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ABSTRACT 

This report summarizes findings of a team of U.S. 
business, labor, and public policy representatives that visited 
Denmark and Great Britain in Fall 1993 to study innovation* in the 
education and training of young people for work. An executive summary 
and introduction are followed by two main sections that present 
findings from the visits. Each section addresses the research 
questions that guided the study. For each question, the report 
summarizes the Danish approach, the British approach, and the lessons 
for the United States. The section on standards and credentials 
focuses on these questions: how standards for vocational education 
and ti aining are set and updated; how standards shape what is taught; 
and what the role of standards is in promoting innovation in 
teaching. The section on assessment covers how the performance of 
learners is assessed and documented and how the quality of vocational 
education programs is monitored and improved. A conclusion extracts 
from these lessons elements of Danish and British vocational 
education that could be used in constructing in the United States a 
viable system for preparing young people to be high performance 
workers of the future: national standards for workplace skills, 
employer "buy-in" of standards development and training, national 
board to oversee the setting and quality control of standards, local 
assessment of student performance, external quality assurance, and 
training and support for teachers. Exhibits from the countries are 
appended. (YLB) 
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The Center for Learning and Competitiveness (CLC) works with American practitioners and 
policymakers to apply the lessons from international education and training systems to policy 
development and system des«m in the United States. Improving the level of workforce preparation is a 
crucial component to improving productivity levels, boosting economic competitiveness and raising 
living standards. By helping US policymakers and practitioners understand the best practices and 
current trends in other countries, CLC helps to ensure that American innovation builds on die 
experience of others and attempts to achieve outcomes of the highest international standard. 

CLC's activities provide access to the people and materials that illuminate the critical principles and 
components of high quality education and training systems. The range of activities include arranging 
targeted study programs of international systems, undertaking strategic consultancies for organizations 
or government departments, leading conferences and seminars in the United States, and publishing 
reports highlighting best practice and innovative methods for system reform. As part ot CLC 
international study programs, American participants meet with their international colleagues and 
counterparts to examine the components and configurations of well-integrated education and Mining 
systems. They gain new perspectives as well as gather specific tools and information that will directly 
strengthen practice in the United States. 

A priority for CLC's work is the dissemination of findings from international investigations to tire 
education and training community, business and union leaders, politicians, journalists and other 
opinion leaders in the United States. CLC also works directly with state governments and with leading 
policy organizations to ensure their reform strategies are shaped and influenced by the experience of 
quality systems in other countries. 

Learning from the international experience has already played an important role in building consensus 
and developing key leadership for nation-wide development of school-to-work transition systems, and 
in providing technical assistance in the establishment of these systems. A focus on the performance of 
international education and training systems enables the United States to learn from other policy 
successes, to avoid reform paths that have been unsuccessful and to ensure that our innovation will 
place us at the forefront of international best practice. 

CLC was founded in 1992 with a three year grant from the German Marshall Fund of the United 
States (GMF) CLC's Executive Director, Anne Heald, created GMF's acclaimed Program on 
Improving U.S. Competitiveness, and has ten years of experience in running influential exchanges 
between me United States and Europe. The distinguished Advisory Board to CLC consists of leaders 
from American political, business, government and union sectors. The work of CLC is also supported 
by other foundations, state and federal governments. Support is also provided by the University of 
Maryland's School of Public Affairs, where CLC is based. 
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PREFACE 



In February 1993, CLC brought together 25 leading experts from state and federal organizations as 
well as international leaders, to identify the most pressing questions and problems that confront 
policymakers and practitioners working to build school-to-work transition systems in the United 
States. The outcome of that meeting was a consensus that there were five areas in need of 
immediate in-depth attention: 



• Building a System: Governance and Finance 

• Developing Standards, Assessment and Credentialing 
Building Partnerships: The Role of Economic Actors 

• Designing Quality Programs 

• Providing Career Guidance 



To address these issues, and with the generous financial support of the German Marshall Fund of 
the United States, CLC initiated its Comparative Learning Teams Project. CLC issued a request for 
proposal nationwide, and respondents were asked to select one of these areas as the focus for an 
international learning investigation, developing levels of inquiry in substantial detail and with 
specific outcomes for their trip. The capacity of teams to effectively disseminate their findings in a 
way that would positively impact on the development of school-to-work systems in the United 
States was a key selection criteria. 

CLC awarded grants to five organizations in the school-to-work transition field who led, planned 
and supported a Comparative Learning Team. The grants enabled each team of at least nine people 
to visit two European sites where sophisticated school-to-work transition systems operate. Each 
comparative learning team participated in carefully planned 12 to 14 day working sessions in 
Germany, Denmark, the United Kingdom, Switzerland and Sweden, where they gained direct access 
to their foreign counterparts and first-hand exposure to European systems. 

The members of the learning teams consisted of leading resource people and experts who are 
catalysts for change in their field at local, state and national levels. Whether they were from the 
private sector, non-profit organizations or government, team members sought answers to the key 
strategic issues facing the development of quality school-to-work transition systems in the United 
States. Hosts in Europe commented on the clear focus of comparative learning team investigations 
around the pressing lessons of importance to American policymakers. A conference held in January 
1994 allowed comparative learning team participants to discuss and refine their reports and findings, 
and to compare observations about international practice. 

Already, the work of the comparative learning teams has had an impact on system-building in the 
United States. Team members were able to build on their European experience when designing state 
systems under the guidelines of the new Federal School-to-Work Opportunities Act. Officials in the 
Departments of Labor and Education, working on school-to-work policies, w by one team 

member about the comparative learning teams project and team members* observations of European 
systems. Participants have spoken at numerous conferences, and published comments in newspapers 
and newsletters. Key findings of the teams are guiding further policy work around key issues such 
as the engagement of industry in school-to-work programs and in the design of skill standards. 
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CLC is now pleased to publish the five reports of the comparative learning teams. Each report 
highlights what the specific team found in their field of investigation, and particularly highlights the 
implications for American policymakers of European experience. We believe that they will be of 
equal interest to those who have examined the European models for workforce development 
previously and those who are being introduced for the first time to international expertise in this 
field. 

For over a decade, American policy leaders have looked to Europe for insight into how to move 
young people effectively from school to the workforce, while providing them with relevant and 
valuable skills. The impressive achievements of European systems triggered much enthusiasm in tliis 
country about the potential positive impact of reform here. Many supporters of school-to-work 
reform in the United States first became excited about the potential impact of reform by looking at 
international best practice and some of the most innovative models of school-to-work transition 
grew out of exploring European sites. 

Now, with the passage of the school-to-work legislation, and with states actively attempting to build 
school-to-work transition systems that will provide widespread opportunities for young people, the 
international experience remains highly significant. Issues that challenge American policymakers in 
building systems, such as developing appropriate funding mechanisms, engaging industry 
partnership and ensuring relevant standards, have long been at the core of investigation in Europe. 
Reform in European systems in recent years reflects current thinking about the delivery of quality 
school-to-work opportunities. 

These reports are timely and relevant for American policymakers who not only want to look at the 
achievements of quality European school-to-work systems, but to explore in more detail the 
elements that enabled such systems to achieve quality outcomes. As states and sites move to 
implement comprehensive reform in the United States under the auspices of the School-to-Work 
Opportunities Act, all five reports will provide valuable information and insight into the best 
international lessons. 

In releasing these reports, CLC would like to thank the German Marshall Fund of the United States 
for their generous support of the comparative learning teams project. We want to express our thanks 
to the lead organizations for the project: The Austin Chamber of Commerce, the New Standards 
Project, The Council of Chief State School Officers, the National Alliance of Business and the 
Northwest Regional Educational Laboratory. 

In particular, we would like to thank the leaders of the five teams who generated such quality 
learning programs for their teams and led the process of developing these significant reports. To 
Bob Glover, Davis Jenkins, Glenda Partee, Esther Schaeffer and Larry McClure, our sincere thanks 
for your dedication and commitment to this valuable learning process. 



Anne Heald 
Executive Director 

The Center for Learning and Competitiveness 
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EXECUTIVE SUMMARY 



The accompanying report details the findings of a team of Americans who visited Denmark 
and Great Britain for two weeks during the Fall of 1993 to study innovations in the education 
and training of young people for work. The team was organized by the New Standards Project 
and was sponsored by the Center for Learning and Competitiveness at the University of 
Maryland. 

The New Standards team mission was to learn about the latest '^aking and practice in selected 
European countries around the use of standards, assessments and credentials in preparing 
young people for work in a global economy. The team chose to study the efforts toward this 
end of Denmark and Great Britain because these countries are in the process of either 
revamping an established system of vocational education and training based on standaixls of 
competence and assessments of performance (as in the case of Denmark) or creating such a 
system from scratch (in the case of Great Britain). In both countries, there is widespread 
interest in how best to prepare young people for productive roles in "high performance 
workplaces," which reflect approaches to organizing and managing work appropriate for 
competing in a global economy. The focus of our study in Denmark and Great Britain was 
primarily on vocational education of young adults who have completed compulsory school, 
which in both countries ends at age 16 or 17, and have either just entered the workforce or are 
preparing to do so. 

The team brought back from its travels a number of lessons that we believe are relevant to 
efforts to establish a viable system of school-to-work transition in the U.S. These are 
summarized in the following. 

Lessons on the Role of Standards and Credentials 

Establish a single body to oversee the setting of skill standards for all occupations and 
industries. 

Both the Danish and the British recognize the importance of establishing national standards for 
vocational education in order to allow for comparability of programs and portability of 
credentials based on those standards. In both countries, there is a unitary structure through 
which such standards are set, thereby ensuring the consistency of standards across programs 
and industries. A key advantage of a centralized standard-setting structure is that it provides 
an accessible forum for business influence over the training system. In the U.S., the highly 
decentralized nature of the mechanisms through which standards and curricula for vocational 
education are developed makes it difficult for business to exercise any real control over the 
content or quality of vocational programs offered by schools and colleges. 

Couch standards in terms of broad goals and frameworks; avoid detailed specifications and 
regulations. Empower those best positioned to affect learning - teachers, trainers, 
administrators and students themselves - to work out the details of how to meet the standards. 
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Both the Danish and the British have learned from experience that only if standards are 
defined in broad terms can a system of vocational education based on such standards have any 
hope of keeping up with the increasingly fast pace of change in the technology and practice of 
work. In the 1960s and 1970s, the Danes followed the approach of specifying skill standards 
for vocational education in encyclopedic detail. In the 1980s, they abandoned this approach, 
having found it to be too inflexible. The bodies that govern Danish vocational education have 
since taken the tack of "steering the system by broad targets and frameworks" rather than 
attempting to control it through detailed standards and regulations. The British have had even 
more recent experience with the deleterious effects of trying to control educational practice 
with standards specified in great detail. 

The approach of using standards to steer rather than control also has a motivating effect on 
teachers. Teachers are able to use their professional knowledge in finding ways to meet the 
needs of individual learners. They are more likely to be committed to their work and to the 
system because their role as professionals is valued. 

Understand that standards by themselves will not produce lasting, systemic reform. 

Standards, by the very nature of the process through which they are set, are bound to lag 
behind the emerging practices and needs of business. It takes a great deal of time, even in the 
most flexible systems, to develop the consensus needed to establish or change a standard. 

The Danish system shows that the standard-setting process in itself doesn't ensure the 
responsiveness of the training system to the needs of industry. The main source of innovation 
and responsiveness in the Danish system is the joint development of curricula at the local level 
by representatives from schools, businesses and labor groups who use the broad goals and 
standards set for the system to guide their work. This on-going collaboration and 
communication on the design and delivery of programs influences the continual refining and 
adaptation of system-wide standards to meet the changing needs of industry. To the Danes, 
the most appropriate use of standards is as signals of broad national goals that guide the 
process of innovation and as means of codifying and disseminating advances in practice. 

Hence, it is not standards per se, but on-going communication in and among all levels of the 
system about what is expected from students that drives innovation and makes for flexibility in 
the system. Standards provide a common language and focus for this communication, but they 
can not substitute for it. 

Begin by building a national consensus about the aims of vocational education. 

In the setting of standards for vocational education, the Danes benefit from the active 
involvement of business and labor that is based on a partnership built up over the course of a 
century or more. Lacking a similar history of involvement in education by industry, the 
British have had to court employers aggressively to become involved in the development of its 
new system of standards and credentials for workplace-based training (the so-called "NVQs"). 
While these efforts have succeeded in encouraging widespread participation by industry in the 
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setting of standards, they have also led to some serious problems, including a proliferation of 
standards defined by the many industry-led bodies, many of which reflect narrowly-defined, 
and often low-skill, jobs rather than broad work roles. In hindsight, these problems might 
have been avoided had the British been more selective in specifying the groups or families of 
occupations and the levels of jobs for which standards should be set. Instead, they leapt into 
the process as though the goal was to set standards for virtually every occupation in the labor 
market, no matter how low-skill or outdated. 

As support builds in the U.S. for the establishment of a system of vocational education and 
training based on industry skill standards, it would behoove us to leam from the British 
experience, which argues for taking a more strategic approach to the development of such a 
system. In particular, the first step should be to build a consensus, based on strategic analysis 
of industry needs and labor market trends, about the purposes the system is to achieve and 
about the methods appropriate to meeting those aims. Like the British, we in the U.S. face the 
challenge of finding a balance between the goals of mobilizing broad involvement in standard- 
setting by industry and generating standards that will truly contribute to workforce 
development. 

Use the restructuring of companies around high performance management practices as a lever 
for encouraging employers to adopt skill standards for training and organizational 
development. 

Active involvement by industry in setting standards for vocational education and training is 
essential it the programs designed according to these standards are to meet the needs of the 
economy for a skilled workforce. 

An increasing number of companies in Denmark and Britain are seeing involvement in 
vocational education of young people and investment in the training of their own workers as 
key elements of a larger strategy for meeting the growing challenge of global competition. 
This larger strategy involves the through-going restructuring of operations around principles of 
management practice promulgated variously under the labels of "total quality management," 
"lean production," or "high performance management." 

In Great Britain in particular, some companies are finding that skill standards are a useful tool 
in their efforts to become high performance workplaces. These companies are using skill 
standards to rethink what they expect of their workers and how they can train workers and 
organize work to enable them to meet these expectations. For national standards to be used 
effectively in this way, efforts must be made to ensure that they reflect the broad demands of 
high performance work - the high skill, high-wage jobs of tomorrow - rather than the 
narrower requirements of the jobs of yesterday. 

This approach of setting standards liigh and then relying on industry, under the force of 
competition, to follow the lead of high performance organizations in using such standards to 
increase the skills of their employees (rather than simply using them to certify skills that 
would be learned on the job anyway) seems to us to be the only way to ensure that the process 
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of setting skill standards has a positive impact on the skills of the workforce and thus on 
economic development The trade-off is that many companies and perhaps whole sectors that 
rely on low-skill, low-wage work will not take part. Still, this seems to be a trade-off worth 
making, since the setting of standards for low-skill work is likely to use up lots of energy 
without generating much value in the way of real learning. 

Question the assumption that academic learning leads to superior transferability of skills and 
knowledge. Ensure that goals and standards for the system reflect a balance between 
theoretical and practical learning. 

Both the Danish and the British are trying to promote the development of vocational programs 
that are occupationally-relevant and yet that allow for transferability of skills when the 
demands of jobs change. Neither has found the definitive answer about what makes for 
learning readily transferable to new and uncertain situations. Still, both are discovering - 
informed by efforts to improve training and organizational learning by high performance 
companies - that the answer lies in finding a better balance between theoretical and practical 
learning. This should cause us to question the assumption, inherent in the design of U.S. 
education at all levels, that academic learning leads to superior transferability of skills and 
knowledge. 

In the best examples from Denmark and Great Britain, learning is organized according to 
projects designed to allow students to tackle practical problems in-depth and motivate them to 
investigate theoretical issues as needed to solve the problems at hand. Guiding the 
development of these programs are goals and standards for the respective systems that reflect a 
balance between theoretical and practical learning. Such a balance needs to be reflected in the 
standards if it is to be manifested in teaching and learning. The British GNVQs represent a 
possible model for achieving the kind of balance that ought to be reflected in standards for 
vocational education in the U.S. In the GNVQ approach, standards for broad enabling 
competencies or "core skills" are defined separately from standards for subject matter and 
technical knowledge and skills. At the same time, the GNVQ standards frameworks require 
that core skills be taught in conjunction with subject area content, thus encouraging the 
integration of applied learning with learning of theory. 

Rely on standards for what students should be able to do, rather than directives of how 
teachers should teach, to drive innovation in teaching. 

The experimentation with methods of teaching that we witnessed in both Denmark and Great 
Britain has been catalyzed not by new directives on teaching, but by changes in goals and 
standards for student learning. For example, Danish teachers are grappling with ways to 
achieve the goal, set forth in the recent reform, of making the education of students more 
"holistic." Scottish educators are finding innovative ways to enable students to achieve the 
outcomes specified in the National Certificate modules. 
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Both countries are finding that competency-based standards - that is, standards defined in 
terms of what students should be able to do tend to promote instruction that is 
interdisciplinary and project-based. So, for example, in the Scottish further education colleges, 
the use in teaching of long-term projects that encompass a range of National Certificate 
outcomes is on the rise. The required integration of core skills units with vocational units in 
the GNVQ framework seems likely to encourage use of interdisciplinary teaching in vocational 
education in England and Wales. 

In both Denmark and Britain, in contrast to the U.S., there is a strong tradition of teachers as 
professionals qualified to decide how best to enable students to achieve standards of 
competence. As part of the on-going reforms in these countries, some excellent guidance 
materials have been produced by government agencies and awarding bodies. But, in general, 
the strategy has been to let schools and teachers in particular figure out how to achieve the 
goals themselves, on the principle that the many problems that will arise are best worked out 
"in the field," rather than in a central office. 

Lessons on the Role of Assessment 

Resist an over-reliance on "end of course" and other on-demand testing; advocate instead 
assessment that is embedded in the teaching and learning process. 

In both Denmark and Great Britain we heard powerful testimony from teachers, workplace 
instructors and students about the pitfalls of excessive use of on-demand testing. The 
prevailing view among vocational educators in these countries is that constantly testing 
students to see how much material they have "absorbed" discourages students from taking 
responsibility for learning. The result is that students educated in such a way are ill-prepared 
to handle the many new and uncertain problems and situations they will confront in work and 
life outside of school. On-demand tests that are set by government agencies and other external 
bodies are even more disruptive to learning because they reflect a distrust in the judgment of 
teachers. 

In contrast, assessment that is an on-going part of the learning process encourages students to 
take responsibility for learning and gives mere authority to make judgments to those who are 
best positioned to evaluate a student's progress: his or her teachers, workplace mentore and 
fellow students. 

The principle of giving responsibility for assessment to students and their teachers as a means 
of motivating learning was reflected in the management practices of the high performance 
companies we visited in Denmark and Britain, where great emphasis is being placed on giving 
responsibility for measuring and ensuring the quality of products and services to all employees, 
including those on the front lines of production or service. Companies such as these are 
relying increasingly on self- and peer-assessment of quality and performance by the workers; 
themselves. It only makes sense that an education system that aims to prepare young people 
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for employment in high performance (and therefore higher wage) companies such as these 
should apply to its modus operandi the fundamental principles that underlie practice in such 
workplaces. 

A related thrust of the on-going vocational education reforms of Denmark and Great Britain is 
the push toward the use of methods that embed assessment in the teaching and learning 
process. There was a general consensus among members of tlie team that a student log, like 
the ones used to monitor progress by Danish apprentices as they move back and forth between 
stints in school and in the workplace, is a powerful tool for assessment and learning. As is the 
practice in the British NVQ and GNVQ systems, such a log could be a key part of a "portfolio 
of evidence" that every student would keep to document what they have achieved and what 
more they need to do to meet the standards. The team was very enthusiastic about the 
portfolio of evidence concept, in part because it places responsibility for evaluating learning 
and growth on the learner, with the teacheror mentor serving as coach and guide. 

Encourage schools to involve members of the community in assessing students. 

Schools and colleges in the U.S. should follow the lead of their counterparts in Denmark and 
Great Britain by involving community members in assessing student work and thus taking 
advantage of the expertise and interest of members of their local communities. This would 
make especially good sense when the work to be judged involves a long-term project - for 
example, a capstone piece of work by older students ~ or portfolios of students* 
accomplishments over time. 

This practice has a number of benefits. The presence of outside judges encourages students to 
do well and gives teachers an incentive to ensure that their students excel. If well run, the 
experience is more than just an assessment, providing opportunities for learning by student and 
teacher alike. As in the final examination of the Danish apprentice, outside judges help 
students to gain insight into their work by asking questions that encourage students to reflect 
on how they accomplished what they did, what problems they encountered and what they 
learned from the process. Teachers benefit by getting a sense of the standards against which 
outsiders judge their students. This practice also provides a more direct way for employers 
and other community members to contribute to student learning than is the case in most 
school-community partnership programs operating in the U.S. today. 

Provide training and support to help teachers learn new methods of assessment; but don't 
expect quick results. 

In 1983, the Scottish Action Plan set forth a bold vision of a system of assessment for 
vocational education based not on external exams, but on assessments developed locally by the 
teacher or trainer who would provide the instruction, assessments that ideally would be 
embedded in the everyday process of teaching and learning. Ten years laten the Scots have 
accomplished a great deal, but they still have a long way to go before there is a level of 
comfort with this approach throughout the Scottish vocational education system. While there 
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has undoubtedly been resistance among teachers and trainers to the new ways of thinking 
about assessment reflected in this approach, the main reason that these innovations have been 
so slow to diffuse is the magnitude of the changes they require in teaching. 

Heeding this lesson from the vanguard work in Scotland, the Scottish and the British are 
generally taking steps to help teachers make the changes in teaching necessary to 
accommodate new approaches to assessment. Still, the British are coming to realize that 
training in assessment procedures is not enough - that to come to a true understanding of the 
expectations set for students by standards and to make reliable judgments of student 
performance against such standards, teachers need to be able to examine and discuss with other 
teachers actual samples of student work. This discussion of student work needs to be 
incorporated as an integral part not just of teacher training, but of the everyday practice of 
teaching. The British are experimenting with ways of doing this through procedures for 
"verification" or auditing by schools and colleges of the results of assessments carried out 
under their auspices. 

Allow teachers to have opportunities for professional development that help them gain a 
clearer understanding of what students need to be able to do to succeed. 

Having clearly stated standards of what students should be able to do to prepare for productive 
roles at work is a critical first step to enabling teachers to make reliable assessments of their 
students. Teachers in the vocational education systems of Denmark and Britain have clear 
standards to guide them in making assessments. Teachers in the U.S do not. They rely 
instead on indirect and often vague signals about what is expected of students from textbooks, 
standardized tests, and packaged curricula. 

However, for teachers to really understand what is expected of students in the world of work 
outside of school, they need to have on-going exposure and contact with that world. One way 
to help teachers stay abreast of new developments in the work world generally is to involve 
professionals from outside the schools in related disciplines and fields in assessing their 
students* work. Teachers would also benefit from working directly in fields related to their 
areas of teaching through internships and sabbaticals. The "practical teachers" in the Danish 
vocational education system spend an average of one week each year working in industry jobs 
in their fields. The British are investing considerable sums to create similar opportunities for 
all teachers, including teachers of academic subjects. 

Don't let the drive for accountability overwhelm learning. 

The mechanism by which the Danes ensure the quality of vocational education is a continual 
communication among parties at all levels of the system focused on improving the system's 
capacity to turn out skilled workers. The Danes do not rely on external examinations to 
determine how well the system is doing toward this end. Such examinations have little to 
offer in the way of useful information about the performance of students generally, especially 
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compared to the judgments of teachers in school and mentors in the workplace who observe 
the student's work daily over a considerable period and who themselves have a clear 
understanding of the standards. 

The architects of the vocational qualification systems under development in Great Britain have 
also chosen, at least thus far, to resist using external examinations as a means of ensuring 
system quality. They see such tests as incompatible with the basic philosophy and structure of 
the systems they are creating, where authority for deciding how to meet the broad goals 
embodied in national standards is given to those at the local level where instruction and 
learning take place. This decentralization of control is believed to be critical for allowing the 
kind of flexibility and innovation characteristic of Danish vocational education and other 
world-class systems of workforce preparation. 

In the U.S., external tests are still seen as the most effective way of ensuring that students and 
schools are performing up to expectations. If anything, pressure for external testing has 
increased in recent years from politicians and members of the public eager to "hold the schools 
accountable." 

If the goal of education is to enable students to take responsibility for learning, the system 
must practice what it preaches. A key lesson from our study of vocational education in 
Denmark and Great Britain is that we cannot hope to encourage students to take responsibility 
for learning if we are constantly giving them tests to see how much they are learning. And we 
can't expect teachers and other educators to take initiative to find ways of improving student 
learning if we are constantly looking over their shoulders. 

Consider an approach to ensuring system quality based on the best of the British verification 
system model. 

The scheme of internal and external verification that is being developed as the main quality 
assurance mechanism in the emerging British vocational qualification systems has much to 
offer a model of vocational education in the U.S., we believe. The model is not that of the 
certification by professional associations characteristic of fields such as health care in the U.S., 
or that of the institutional accrediting bodies familiar to U.S. higher education. The standards 
typically used for accrediting educational programs in these and other quality control systems 
common in the U.S. are based typically not on measures of outcomes of student learning but 
instead on institutional characteristics that serve as proxies for the quality of teaching and thus 
of learning, such as credentials of the faculty, number of books in the library, duration of the 
course, course structure, content, etc. An army of part-time accreditors is employed to ensure 
that institutions meet these criteria, which bear a weak relationship to quality student 
outcomes. 

In the model suggested by the British approach, quality control is focused on outcomes of 
student learning, and in particular on the evidence by which outcomes are assessed. This gives 
teachers and others at the level closest to the student broad flexibility and authority to decide 
how to help students meet the standards. This sort of flexibility is constrained by accreditation 
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systems common to the U.S., which focus on inputs and content of instruction. One member 
of our team warned that, "The minute we in the U.S. start to say, 'make sure that they are also 
teaching such and such, and make sure that they do things this way, 1 that is when we start to 
get into trouble." 

To avoid the patronage and parochial interests that characterize certification and accreditation 
bodies in the U.S., a system of quality assurance for workforce education would ideally need 
to be coordinated by a single national body that transcends industry or professional affiliations. 
In Denmark, this is the role of the Coordinating Council for Vocational Education, which 
includes representatives from education, business, labor and government. In Great Britain, 
such a role is played by SCOTVEC for the Scottish vocational qualifications, and NCVQ for 
the U.K.-wide NVQ and GNVQ systems. Without such an umbrella organization in the U.S., 
it will be impossible to set national standards, let alone ensure that national standards are being 
met. 

In Denmark and Great Britain, we saw the promise of a standards-based system of vocational 
education and training that is successful in preparing young people for work in a global 
economy. Where the systems of these countries work best, a critical factor is the involvement 
of employers, whether through the training of apprentices, contributing to the development of 
standards or curricula, or using standards to improve the skills and performance of incumbent 
workers. Employer involvement is no pana* 3a. Employers do not speak with one voice and 
at times give conflicting messages, for example, by advocating that education should focus on 
"employability and adaptability," but then hiring the classically-trained, narrowly-proficient job 
candidate. As the final customer of workforce education and training, however, employers 
must play a central role in developing and sustaining a system of workforce preparation. Still, 
employer leadership should be cultivated in a way that facilitates communication and 
cooperation at every level of the system -- national, state and local - between all stakeholders, 
including workers, educators and policy-makers. Our study of vocational education in 
Denmark and Great Britain has persuaded us that it is only on such a foundation of 
communication and cooperation that we in the U.S. can hope to build a system for preparing 
young people to be high performance workers of the future. 
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INTRODUCTION 



This report summarizes the findings of a team of Americans who visited Denmark and Great 
Britain during two weeks in the Fall of 1993 to study innovations in the education and training 
of young people for work. The team was organized by the New Standards Project, a 
partnership of states and school districts that are seeking to achieve dramatic improvements in 
U.S. education by setting clear, high standards for all students and by developing methods of 
assessment designed not only to measure but also to motivate student learning. The members 
of the team are listed in the Appendix. Each of these individuals is involved in initiatives 
seeking to enhance U.S. economic competitiveness by improving the education and training of 
Americans from the primary grades onward. 

The New Standards team was one of several "learning teams" sponsored by the Center for 
Learning and Competitiveness of the University of Maryland. Each team received a grant 
from the Center to spend two weeks in European countries of its choice investigating a 
particular aspect of the approaches used in those countries to prepare young people for work. 
The purpose was to bring back models and lessons based on European experience that would 
help to further efforts in the U.S. to create a viable system for school-to-work transition. 

The New Standards team chose to focus on the use by European countries of standards, 
assessment and credentials in preparing young people for work. In the U.S., there is 
widespread recognition of the need for a coherent system to prepare young people who do not 
go directly to four-year college out of high school for entry into the workforce. There is also 
a growing consensus that the strategy for creating such a system must include, first, high 
standards stated in terms of the ctmpetencies a young person needs to have to become a 
productive worker and, second, assessments of performance in relation to those standards that 
also promote learning. 

The New Standards team chose to study Denmark and Great Britain because these countries 
are in the process of either revamping an established system of vocational education and 
training based on standards of competence and performance assessment (as in the case of 
Denmark) or creating such a system (as in Great Britain). 

In Denmark, the main focus of our analysis was on the changes in what is called upper- 
secondary vocational education following a 1991 law calling for reform of the system. Of the 
ninety percent of Danish students who go on directly for further education following 
compulsory school (which ends at age 16), about two thirds enter vocational education, either 
in technical trades or in commercial fields. The other one-third goes to gymnasiums where 
they prepare for entrance to a university. Vocational education programs are organized 
according to alternating periods of training in school and in the workplace. Most programs 
last from three to five years. Standards for vocational education are set by trade committees, 
which are made up equally of representatives of industry associations and labor unions. These 
trade committees also play a central role in the governance of technical colleges where the in- 
school portion of vocational education takes place. 



Center for Learning and Competitiveness 

The Role of Standards, Assessment and Credentialing 



ERIC 



17 



The 1991 reform called for changes in two main areas: instruction; and governance and 
finance. In the area of instruction, the reform sought to broaden vocational education to 
ensure that young people are prepared for a modern workplace characterized b> increasingly 
rapid change and also have the flexibility to pursue further education and training beyond the 
upper secondary level. Business and labor, through the trade committees, sought to remedy 
the proliferation and over-specialization of technical education programs by consolidating the 
number of such programs from over 270 to under 90 and broadening the definition of the 
standards that guide instruction in these programs. The latter push especially stemmed from 
the recognition on the part of business and labor that no longer can a worker be guaranteed 
lifetime employment in a particular occupation. In advocating broader standards that stress the 
importance of "learning to learn" and adaptability, both business and labor sought to change 
workers' awareness of what it will take to succeed in the workplace of the future. The reform 
also required that "vocational education should be holistic," with the aim of encouraging 
instruction in school to be more like learning in the work world » more interdisciplinary and 
applied - so that students can learn how to deal with problems and situations like those they 
will encounter as workers. At the same time, the reformers sought to inject more academic 
rigor in vocational education by increasing the number of required academic courses and 
introducing an external examination of academic subject matter to be given at the end of the 
first year. Nevertheless, the reform preserved the basic structure of vocational programs, in 
particular the "sandwich model" with periods of training alternating between in-school and on- 
the-job. 

In the area of governance and finance, the reform sought to increase the quality, efficiency and 
market responsiveness of vocational education. It advocated "steering by targets and 
frameworks" instead of "controlling by detailed regulations" as the best way to govern the 
system. It established the technical colleges as "self-governing" institutions with boards 
representing the college's main customers in the local community. At the same time, it moved 
the funding of the technical colleges to a system where money follows the student, thus 
increasing competition among the colleges. 

In Great Britain, we focused on recent efforts to establish new systems of vocational 
credentials or "qualifications." One set of such qualifications, the National Vocational 
Qualifications, or NVQs, are nationally-recognized certifications of competence in specific 
occupations. They are designed to provide certification of skills acquired primarily on-the-job. 
Standards for NVQs are set by industry associations or "lead bodies." To receive an NVQ 
award, a candidate must assemble a "portfolio of evidence" demonstrating that he or she is 
competent in the skills specified by the standard. The candidate's supervisor reviews this 
evidence and relies on performance criteria included in the standards to assess whether or not 
the candidate has met a particular standard. 

The British government launched the NVQs in 1986. In doing so, it had several motives. 
First, it sought to create a meaningful credential for young people involved in "Youth 
Training" programs, which offer subsidized employment and training to disadvantaged youth. 
These programs, which were heavily funded under the Labor governments of the 1970s, had 
been widely criticized for providing employers with publicly-subsidized youth labor and 
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offering young people little in the way of useful experience and training and no credential. 
Second, the government hoped to promote continual learning by experienced workers. Here 
the idea was that employers could use the national skill standards for particular occupations 
afforded by the NVQs to examine the kinds of skills they need in their employees and to 
rethink training and work organization appropriate to. enabling employees to acquire these 
skills. Finally, the government sought to consolidate and revitalize existing skill certification 
systems, which were highly fragmented and, outside of a few craft occupations, were not held 
in high esteem. The government chose to offer NVQs through the awarding bodies of these 
existing certification systems, but at the same time established the National Council for 
Vocational Qualifications (NCVQ) to oversee and coordinate the work of these bodies. 

Since the first NVQs came on line in 1988, a great deal of energy has been put into their 
development. As of December 1992, standards have been specified for over 80 percent of 
occupations in the British labor force. Thus far, over ?i0,000 have received an NVQ 
qualification, and an estimated one million workers are working toward one. Given that NVQs 
really only came on line in 1988, this represents a promising level of participation in a 
workforce of 23 million. Although still primarily used for youth training, NVQ awards are 
also now being pursued by an increasing number of adult workers as well. 

The British are also in the process of developing another set of vocationally-oriented 
qualifications called the General National Vocational Qualifications, or GNVQs. The GNVQs 
provide a framework for school- and college-based vocational education of students in the 16- 
19 age group. Unlike the NVQs, the GNVQs are designed to prepare young people for entry 
into broad occupational streams, not to train them for specific jobs. GNVQ awards are 
therefore offered in a few broad categories that include manufacturing, leisure and tourism, and 
health and social care. Moreover, the GNVQ framework requires the integration into 
instruction in these occupational areas of "core skills," broad enabling skills such as 
communication, application of number, and use of technology that are intended to increase the 
breadth of learning in GNVQ programs. 

Proposed by the British government in 1989, the idea for the GNVQs was advocated by the 
Confederation of British Industry, a group not unlike our Business Roundtable, in its 
influential 1990 report entitled "Toward a Skills Revolution." This report gave voice to a 
growing concern among leaders in British industry and government over a number of trends: 

• Increasing enrollment in education beyond the compulsory age of 16, which has 
shot up from around 42 percent oi the cohort in 1981 to 70 percent today (this 
includes enrollment in further education colleges - the "British community 
college") - the concern here is how to accommodate the growing numbers of 
young people seeking further education with programs suited to their interests and 
needs; 

• Increasing disenchantment with the A-level system of university preparation as too 
academically-oriented and overly exclusionary; 
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• Recognition of the inadequacy of the NVQs for providing broad-based preparation 
needed by young people entering the workforce. 

Because of the strong support for the idea, GNVQs were put on a very short development 
timetable. In 1991-92, GNVQs were piloted at 20 schools and further education colleges 
throughout Britain. Last year, the number of pilot sites was increased to 100. This year, 
1993-94, GNVQs are being offered throughout England and Wales. Recently, the government 
announced that a GNVQ level 3 qualification will be equivalent to two A-levels, the minimum 
requirement for university entrance. Still, there is not wide-spread acceptance, especially 
among university circles, that this equivalence is valid. 

Many of the features of the NVQs and GNVQs were pioneered in work on creating a new 
approach to vocational education in Scotland beginning in the early 1980s. In 1983, the 
Scottish Action Plan put forth a bold blueprint for a system of vocational education based on a 
number of key principles, including: 

• Modularization of curriculum, to allow maximum flexibility in constructing 
programs to meet the students' needs; 

• Learning outcomes stated in terms of standards of competence; 

• Reliance on "internal assessment" by the teacher or instructor (as opposed to 
external assessment through standardized exams) based on performance criteria 
specified in the learning outcomes. 

The Action Plan recommended rationalizing all existing vocational qualifications in Scotland 
under a single system of National- Certificate modules based on these principles. In 1984, the 
Scottish Vocational Education Council (SCOWEC) was established to serve as the single 
awarding body for the National Certificates. The development of National Certificate modules 
proceeded at a rapid pace. The first modules developed were designed primarily as a 
framework for vocational education for young people in the 16 to 18 age range through 
programs that would be offered primarily, though not exclusively, in further education 
colleges. Later, the Higher National Certificate and Higher National Diploma programs, 
postsecondary technical programs that had existed before the Action Plan, were converted to 
modular, competency-based format and put under the SCOTVEC bailiwick. Also, as NVQs 
have come on line, SCOTVEC has developed modules based on the standards developed for 
the NVQs by the U.K.-wide industry lead bodies. These Scottish equivalents of the NVQs are 
called, appropriately enough, SVQs - Scottish Vocational Qualifications. 

Underlying the reforms of vocational education in both Denmark and Great Britain is a search 
for better ways to educate young people to be productive and prosperous workers in a global 
economy. The main challenge is, given the increasing pace of change fueled by global 
economic competition, how to create a program of learning chat will prepare young peopie for 
entry into specific occupations, yet give them the capacity to leam new skills and take on new 
roles as the demands of the workplace change. 
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The methods for meeting this challenge that these countries are exploring in their education 
systems mirror strategies being adopted by companies and other organizations throughout the 
industrialized world that are successfully responding to the rapid change wrought by global 
competition. These "high performance workplaces" are meeting the challenge of change by 
creating within their organizations a stable mechanism for continuous learning and innovation. 
A growing number of companies in Europe and throughout the world are achieving this by 
transforming their organizations according to international standards for management practice. 
ISO 9000, the standard for European companies, is increasingly being adopted by firms in the 
U.S. 

For this reason, plus the fact that workplaces outside of school can add a rich dimension to 
vocational education, we decided to include in our itinerary visits to companies as well as to 
schools and community colleges. 

In the remainder of this report, we present our findings from these visits. The findings are 
divided into two main sections: Standards and Credentials; and Assessment. Each section 
addresses the research questions that guided our study -- questions that confront the efforts 
each of us is involved in to improve education and training of Americans for work. For each 
question, we summarize the Danish approach, the British approach and the lessons for the U.S. 
In the conclusion, we extract from these lessons elements of Danish and British vocational 
education that we think could be used in constructing in the U.S. a viable system for preparing 
young people to be high performance workers of the future. 
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STANDARDS AND CREDENTIALS 



1. How are standards for vocational education and training set and updated? 

Main issues of interest to the team: 

• What forrn do these standards take?; 

• How to get industry involved in the setting of standards; 

• How to ensure that standards keep pace with rapid changes in the technology and 
practice of work; 

• How to encourage employers to use standards and credentials for vocational 
education in hiring new employees and in training incumbent employees. 

Danish Approach 

A System Built on Cooperation 

Standards for vocational education in Denmark are developed through close cooperation 
between the Ministry of Education, employer associations, labor unions and technical colleges. 
Exhibit 1 illustrates the structure of the system through which this cooperation takes place. 

Together, business and labor have the greatest influence on the shape of Danish vocational 
education through joint representation on governing bodies at three levels: 

Council for Vocational Education - Deals with the overall governance of the 
vocational education system, advising the Minister on matters such as the goals and 
structure of the system; approval of new trades, schools, and programs; teacher 
qualifications; and entry requirements. This Council is established by the Minister of 
Education, who appoints the chairman. Its membership is dominated by business and 
labor representatives, but includes representatives of the teachers' unions and technical 
colleges. 

National Trade Committees - Made up of equal numbers of employer and labor 
representatives, these committees determine (subject to formal approval by the 
Ministry): the duration of training, required courses, objectives and general content of 
courses and assessments. 

Local Education Committees - These local affiliates of the National Trade Committees 
advise the technical colleges on programs in their trade area and assist in finding 
placements for students in local firms. They are akin to the industry advisory groups 
set up by community and technical colleges in the U.S., but the extent of their 
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involvement is in general greater and they are better organized, with strong support 
from their national organizations. 

The active involvement of business and labor as partners in setting standards for vocational 
education - indeed the Danes refer to business and labor as "the partners" - made a big 
impression our team, coming from the U.S., where cooperation between business and labor is 
uncommon and joint involvement in formal education is even rarer. 

The influence of the partners over the system is not without checks and balances, however. 
For example, while the Ministry for Education defers to the authority of the partners to decide 
the substance of vocational programs, it is the Parliament that determines the level of funding 
for vocational education. This means that, in setting the standards for the system, the partners 
must take into account the level of resources available to achieve the aims reflected in the 
standards. This encourages a healthy dialogue between the partners and government which 
helps to ensure that resources are not indiscriminately spent. 

What form do the standards take? 

The form taken by the "standards" differs depending on which level of the system they 
originate from. From the Council for Vocational Education and Ministry of Education come 
broad directives about the structure of the system and goals to which it should aspire. The 
goals set at this level have an important influence on the direction of efforts at all levels of the 
system. For example, a key goal set forth in the 1991 law for the reform of vocational 
education was that "education should be holistic." As will be discussed below in the section 
on the impact of standards on teaching methods, there has since been a flurry of activity at the 
level of the trade committees and, perhaps even more so, in the technical colleges and 
company programs where instruction takes place, aimed at figuring out how to realize this goal 
in practice. 

The "standards" developed by the national trade committees are really broad curriculum 
frameworks. Exhibit 2 contains excerpts from the framework for the field of mechanical 
engineering, which covers the specialties of machine tool fitter, plastics mechanic, ship's fitter, 
turner, cutter and bench worker. (Note that the part of the frameworic excerpted in the Exhibit 
2 details the structure and content of those parts of the program that take place in school; the 
structure, content and objectives of the "practical training periods," which take place in firms, 
are outlined later in the document.) As is evident from these excerpts, the framework is quite 
specific about the str cture, duration, and content of the educational program, going so far as 
to indicate which school subjects and technical skills should be taught in which period of the 
program. This and the other frameworks we were shown differ from "content standards" as 
they are generally thought of in the current standards-based reform movement in the U.S. in 
that they are much more specific about what should be taught. 

However, a key feature of the Danish frameworks is that they also specify what the student 
should be able to do after having completed a given piece of the program (or the program as a 
whole). It is in this respect that the Danish approach to vocational education can be 
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considered competency-based. For example, page 2 of Exhibit 2 indicates that in the Second 
School Period, through instruction on the thermal joining of construction parts, "The student 
shall be able to carry out hard soldering in steel and copper materials." It is left to the 
technical colleges and companies to design the syllabi that detail what specifically students 
should learn in a particular course to enable them to achieve the outcomes specified in the 
framework. And, in keeping with the strong tradition of teacher authority or professionalism 
in Danish education generally, it is the responsibility of the teacher or trainer to decide what 
methods of teaching to use for this purpose. As we were told in the Ministry of Education, 
"Knowing how to reach the goals is what it means to be a teacher." 

While the frameworks provide general guidance on the level or quality of student performance 
that is sufficient to meet the standard in a particular area of content, ultimately, it is left to the 
teacher or trainer to determine the standards of performance. The reliability of judgments 
made by teachers and mentors derives at least in part from their common status as skilled 
craftspeople. The hallmark of the skilled craftsman is the ability not only to apply the skills of 
the trade with an acceptable level of skill, but to recognize craftsmanship in others. When we 
asked a foreman at a company we visited in Copenhagen how he knew whether an apprentice 
has met a given standard, he simply said: "A skilled person knows when an apprentice has 
done a task well or badly." The reliability and validity of judgments made by teachers in the 
school are also enhanced by the on-going communication that takes place between teachers in 
school and mentors in the workplace. 

While the teacher and mentor have broad authority to decide how to teach and how well the 
student needs to perform to meet the standard, the framework provides rather specific guidance 
on what the student should be learning at various points in the program. The document from 
which Exhibit 2 is taken is 45 type-written pages, and that only includes excerpts translated 
into English. We were told that the frameworks for Danish vocational training used to be 
specified in much more detail. According to J0rgen Andersen of the Danish Metalworkers 
Union, in the past the Danes defined standards using a job task analysis methodology that by 
Andersen's description sounded to us much like that used in the U.S. by the V-TECS 
Consortium and others. The Danes produced volumes containing these standards that were 
hundreds of pages long, not just tens of pages, as they aie now. This approach was 
abandoned, Andersen says, because it could not keep pace with rapid changes in workplace 
technology. The move since has been toward frameworks specified in terms of more broadly 
stated student outcomes. The 1991 refonn provided further impetus in this direction, by 
including as one of the chief goals, "to encourage steering of the system by targets and 
frameworks instead of detailed regulations." 

How adaptable is the system? 

The team was impressed by the level of cooperation among the various parties in the setting of 
the targets and frameworks for vocational education. Still, it seemed to us that, given the time 
it takes for parties with different interests to reach consensus, the system might not be able to 
respond to rapid changes. 
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What we discovered was that, while the national frameworks may lag behind the state-of-the- 
art of practice in industry and the schools, this is not considered to be a major problem The 
IZZ £*!!! Vf* xl S S6en 35 S6tting a relatively Wgh «P«tation of the minimum students 
Uie fram^w'orks * ^ * g ° W6 " bey0nd 1116 StandardS specified 1x1 

The fact that it is up to teachers and mentors to decide the specifics of what is taught and how 
it is taught means that the local level is the locus of innovation in the system. Perhaps the 
major source of innovation is the students themselves, in that they provide a communication 
hnk between school and workplace that allows both sides to stay abreast of changes in the 
other. Another source of innovation at the local level is the contract training courses that the 
technical colleges offer to local employers, which give the colleges a window c i the latest 
technologies and techniques in industry. According to Jens Erik Simonsen, Head of Industrial 
Automation at Arhus Technical College, "Most of the changes we are making in the basic 
vocational programs in Automatic Mechanics are coming mainly from what we are learning 
through the further education we offer companies." Erik Torp Nielsen, head of Arhus 
Technical College's metalworking program sees the college's involvement in customized 
training for firms as vital to the quality of its apprenticeship programs. He believes that 
practical instructors who teach at institutions that are not engaged in training and R&D work 
for firms have a difficult time keeping up with advances in their field. 

i°^T^ t[ T TV* fl ° W Wdl fr ° m ** l0Cal level U P variou * ^vels of the 

system. The local trade committees are evidently effective in keeping their national 

organizations informed of developments in the field. The people we spoke with in the 

Ministry and national level partner bodies seemed to be very much in touch with what was 

going on at the local level. The standards serve as a common understanding that facilitates 

Phrr™!!™ 21110118 ? Vari ° US l6VelS 0f the s y stem - Yet ' il is not *e standards that drive 
changes in die system, but rather the communication itself - communication among people 
who understand the standards. pwpic 

By the time an innovation is formally codified in one of the frameworks - Exhibit 3 
illustrates the process for an entire new program - it has been tried out at the local level and 

HyT?^? , UP t ? f0U ^ ±& SySt6m - The COdified Standards serve 35 benchmarks that are 
used to ratchet up improvements in the system as new ideas are tested out and formalized as 
policy. Innovative programs in the colleges and certainly the training offered by most 
progressive companies will go beyond what the frameworks require in terms of what thev 
expect students to be able to do. By the same token, the standards compel others - 
particularly smaller companies - to do more in their training than they omerwi.se would. So 
the effect is to keep the average level of practice in the system high. 

There is still some concern among members of the team that while die speed with which 
kSflfh f ™ ewoi1cs are made within an industry may be good, the fact that the system 
is structured by trade committees may create walls between trades diat make it difficult to 
create new programs in occupations that cut across families of trades. Nevertheless, the fact 
that under the recent reform, the number of occupational families for which there are separate 
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frameworks was reduced from some 270 to under 90, despite the turf battles and entrenched 
constituencies that had to be overcome, suggests that this is not an insurmountable problem for 
the Danish system. 

High Level of Employer Involvement 

In general, Danish employers need little encouragement to value the products of vocational 
education because they are so heavily invested in the system that these products are in a sense 
"their own." When representatives of Danish employer and employee associations we met 
with talked about the technical colleges, they referred to them as "our schools.' 1 This attitude 
may stem from the fact that the vocational education system is funded at least in part through 
corporate taxes that companies must pay whether or not they train apprentices. Employers 
therefore take pains to ensure that the technical colleges and the vocational system generally 
meet their needs by participating in the governance structure through which standards are set 
and on the boards of the technical colleges, and by providing training slots for apprentices in 
their workplaces. 

This is not to say that there is universal support for vocational education among Danish 
employers. Not all firms train apprentices, and some companies, many of them small 
employers, have opposed the move toward higher standards and broader skill requirements, a 
trend accelerated by the 1991 reform. According to Jens Peter Jensen of Arhus Technical 
College, these companies prefer more narrow training in skills relevant to particular jobs. 
"These companies think that they can do jvst fine without 'smart workers.' For this reason, 
some do not hire apprentices." Jensen and his colleagues at the college see as part of their 
mission educating employers like this about education and training appropriate for superior 
economic performance. Says Jensen, "The college is trying to tell these small business 
owners, 'If you are changing your products so often, then the skills required of your worker 
are also changing rapidly. So, you should be training your workers more broadly. 1 " 

None of the companies we visited in Denmark needed this kind of lesson. Each is involved in 
the training of apprentices and is committed to vocational education as an effective means of 
maintaining the supply of skilled workers. Moreover, each is heavily invested in continual 
training of all of its employees, not just its skilled workers. 

These companies see their human resources as the sole source of competitive advantage not 
only for themselves but for Danish industry generally. These are the companies that, along 
with labor unions seeking to improve the flexibility of workers to change jobs in a turbulent 
economy, are pushing for higher skill standards for education. To these companies and 
unions, there is really no viable alternative in a tiny country with no resources for economic 
development other than the talents and energies of its people. The fall ot the Berlin Wall has 
strengthened resolve of Danish industry to pursue a high skill, high wage strategy of economic 
development. According to Jens Agerley of Radiometer, "Industry wants the government to 
set higher and higher standards for education and R&D since we can't compete on the basis of 
wages with Eastern Europe." 
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In every one of the firms we \ sued, investment in training and human resource development 
is a key part of a larger approach to management aimed at improving the quality of products, 
hastening the speed of innovation, and improving customer satisfaction. All had attained 
certification under ISO 9000, the international standard for total quality management. AH were 
convinced that undertaking the sometimes dramatic restructuring required to meet the ISO 
9000 standards was absolutely necessary if they hoped to have access to foreign markets, and 
certainly markets in Europe. Although we only visited a handful of firms in Denmark, we got 
the impression that the restructuring of Danish companies around ISO 9000 or total quality - 
what in the U.S. is sometimes referred to as the move to "high performance work practices" — 
is widespread among Danish companies. Given the strong reliance of the Danish economy on 
exports - the manufacturers we visited shipped 90 percent or more of their products outside of 
Denmark - this is likely to be the case, especially if such high performance practices are a 
pre-requisite to remaining competitive in international markets. 

This trend toward greater investment in training as part of an overall move to high 
performance management is not confined to the largest companies. Indeed, commitment on 
the part of management, rather than size or resources, seems to be the key to making this 
transformation happen. One of the firms we visited, COTAS, a manufacturer of computer 
controllers for industrial processing machines that has only 45 employees, underwent a 
dramatic transformation after its owners/managers decided to go for ISO 9000 certification. 
Among the major changes at COTAS, according to Managing Director, Ole Riis Hansen, is in 
the area of training. Before ISO 9000, COTAS did not do much in the way of training for its 
employees; now every employee has a regularly-updated education plan, and "training of 
employees is going on all the time." Among the other things that COTAS now does 
differently under ISO 9000 are: 

• Every worker is encouraged to get experience with every stage of production, in 
part to ensure that workers can fill in for each other, but even more to engender a 
broader understanding among all workers of the company's products and 
production methods. 

• Every function is a unit that has control over the quality of its output. There is no 
more "double-checking" of workers' work, other than the final testing of each 
product 

• R&D (which involves nearly half of COTAS 's 45 employees) is located on the 
production floor and production workers are consulted on the design of new 
products at each stage of the process. 

The movement of Danish firms toward ISO 9000 and high performance work is starting to 
change the face of vocational education in Denmark. Certainly it is having an impact on the 
practical training of apprentices who have training placen . its in firms that are adopting high 
performance practices. At COTAS, apprentices are trained through rotation among operations 
at every stage of production, from design to manufacture and shipping. At Danfoss A/S, a 
maker of high quality metal valves, apprentices participate as members of teams that have 
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responsibility for the entire production line for a given product. At Radiometer, apprentices, 
like all employees, are expected to suggest ways to improve quality and efficiency. 

We saw no evidence that these changes in the practical training of apprentices have yet been 
reflected in the frameworks that govern vocational training in specific trades. We suspect that 
it will only be a matter of time before this happens. One indication is that the philosophy of 
high performance management is becoming evident in the policy and governance of the 
vocational education system as a whole. Two of the major goals for vocational education 
promulgated by the 1991 reform clearly reflect fundamental principles of high performance 
management First, the goal of steering by targets and frameworks instead of detailed 
regulations reflects the principle of creating constancy of purpose through broad goals that the 
members of an organization can believe in and work toward. Second, the goal of further 
decentralizing to the technical colleges control of the system reflects the principle of 
empowering those at the front lines of production or service to take responsibility for ensuring 
quality and making improvements to the system. 

British Approach 

Great Britain offers an opportunity to examine standard setting in a milieu more like the U.S. 
than Denmark in that there has not been a history of active involvement of business or labor in 
vocational education, outside of a few industries with a tradition of apprenticeship. Three 
different approaches to involving industry in the setting of standards are reflected in the 
National Vocational Qualifications (NVQs), the General National Vocational Qualifications 
(GNVQs) and the SCOTVEC National Certificate. 

NVQs: Turning Standard-Setting Over to Industry 

To encourage industry ownership of these occupation-specific qualifications, the government 
gave responsibility for setting and updating the NVQ standards to industry and professional 
associations called "lead bodies/' Most of these lead bodies had been part of existing systems 
of training and certification, which the government sought to consolidate and revitalize under 
the single NVQ system. 

Since the NVQs were launched in 1986, the government has provided funding to the lead 
bodies to define skill standards for occupations in their respective domains using a common 
methodology. A key principle of this methodology is to define competence from the top 
down in terms of the broad functions of a particular occupation rather than from the bottom up 
in terms of the myriad tasks a person carries out in the course of doing a job. The goal is to 
avoid the over-specification characteristic of traditional job task analysis. 

This "functional analysis" methodology works as follows. Expert practitioners of a particular 
occupation are convened and asked to identify the broad purposes and key functions of their 
occupations. The key functions are defined as units of competence and are further broken out 
into elements of competence. An example of the "functional maps" that result from this 
process is given in Exhibit 4. For each element of competence, the focus group of 
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practitioners is asked to define performance criteria that describe behaviors indicative of 
competent performance of the particular element of competence and that could therefore be 
used by an assessor in judging whether someone is competent in the particular domain. 
Rounding out the definition of each element of competence are "range statements," which give 
a sense of the range of contexts or situations that a competent person should be able to handle, 
and "underpinning knowledge," which is the basic knowledge required for competent 
performance. Exhibit 5 shows the definition of a particular unit of competence for a baker. 

The work of defining NVQ standards was slow to get off the ground in the three years 
following the introduction of the first NVQs in 1988. Part of this was due to resistance or at 
least indifference of employers to getting involved, but it also reflected a lack of experience of 
many of the lead bodies with the functional analysis methodology. In the last couple of years, 
the level of involvement and comfort among the lead bodies with standard-setting has grown ' 
considerably. Industries such as retail and hospitality, which had never done much in the way 
of training or certification, are now taking a leadership role. As of December 1992, standards 
had been specified for 80% of occupations in the British labor market. Thus far, over 350,000 
have received an NVQ qualification, and projections are for over one million qualifications by 
1995, with the number expected to increase exponentially thereafter. 

Despite the recent spurt in NVQ activity, the setting of NVQ standards suffers from a number 
of shortcomings. One problem is that there is a fair amount of overlap in the standard being 
developed by different lead bodies. This is not surprising given that there is overlap among 
the lead bodies themselves. For example there are at least three lead bodies for engineering- 
related occupations. 

A related problem is the proliferation of NVQ qualifications, each of them in a specific, and 
sometimes obscure, area of specialization. For example, there are three separate level two 
qualifications for "installing and commissioning telecommunications terminal equipment": 
"cellular," "fixed," and "radio." NVQs are being developed at five levels in each of a 
spectrum of occupational categories, which are listed in Exhibit 6 (the "NVQ Framework"). 
Note that higher NVQ "levels" are characterized by greater breadth and range of competence, 
ability to cope with non-routine activities, ability to organize and plan work, ability to assume 
responsibility and work without supervision, and ability to supervise others. The proliferation 
of standards has produced a prodigious amount of paper. For example, a thick three-ring 
binder is required to contain all the standards for "Staff Working with the Under Sevens and 
their Families" (i.e., day care workers). This despite the fact that in opting for the functional 
analysis methodology, the British sought to avoid the voluminous specification of skills and 
tasks characteristic of the traditional job task analysis. 

At root of these problems is not the functional analysis methodology per se but the fact that 
the British launched the NVQ system without conducting labor market analyses of the kinds of 
occupations for which skill standards should be defined and without developing an over-riding 
vision that would help to foster involvement and consensus by employers in the standard- 
setting process. In its eagerness to get employers involved in standard-setting, the government 
gave responsibility for the process to existing industry and professional bodies or to newly 
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formed coalitions of such groups. The result was that the standards reflect not only the 
proliferation and redundancy of these lead bodies - there are currently over 150 of them - but 
also the often antiquated occupational structures they represent. 

In fairness to the British, this approach may have been the most expedient given the perception 
of crisis in the mid-1980s over the large number of young people who were entering the 
workforce directly out of compulsory school with few skills, a trend that was seen as 
threatening the country's ability to compete in global markets. The NVQs were proposed as a 
way of providing these young people with a structured approach to learning on the job and 
with a credential designed to motivate them to pursue learning further whether on-the-job or 
through more formal schooling. Indeed, British industry seems to have bought into what is 
clearly a rather bureaucratic system at least in part because of the promise it held for 4 • 
improving the skills of the growing low skill segment of the workforce. Moreover, there may 
have been no other choice but to turn to existing industry groups in mobilizing employer 
involvement in NVQ standard-setting and support for NVQ qualifications. 

However, the upshot of handing over standard-setting to industry has been the development of 
standards that in general seem to us to be too focused on narrow occupation-specific 
competencies and too oriented toward low-skill jobs. Noticeably absent from the NVQ 
standards is attention to broader competencies of a sort critical to the ability 10 learn new job- 
specific skills, for example, communication, problem solving, and use of technology. This 
casts doubt on the appropriateness of the NVQ system for youth training - one of the main 
purposes it was designed to serve. Indeed, we wonder how appropriate NVQ standards 
(especially those at the lower levels) are for guiding the training even of incumbent workers, 
for whom, given the increasing pace of technological change, the ability to leam is as 
important as for new entrants to the labor force. Our misgivings were supported by a number 
of employers we met with, As one of them said, "The lead bodies are setting standards not 
for the jobs of tomorrow, but for the jobs of yesterday." 

The narrow definition of NVQ standards around occupation-specific competencies is not 
surprising given that in the NVQ standard-setting process there is nothing to counter-balance 
the interest of employers in providing only as much training as is necessary to enable workers 
to do their current jobs. Although the NVQs may offer the British worker a means of 
recognition for the skills he or she acquires on the job where before there was none, it is not 
clear to us how much they benefit the worker by giving incentives to their employers to 
provide training that would not otherwise not be offered. 

A company training manager who has long been involved in vocational education in Scotland 
argued that the government's strategy for implementing the NVQ system has suffered from 
conflicting aims. On the one hand, the government wants NVQ standards to have a positive 
impact on workforce development and economic performance. On the other hand, it wants to 
ensure broad participation in the process by industry. These aims are in conflict because the 
former relies on high standards and yet the latter leads to a watering down of standards. This 
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manager believes that the government should work to ensure that standards remain high. 
Competition, not coaxing or coercion by government, will compel companies to make 
productive use of high skill standards, he believes. 

Perhaps an unanticipated benefit of the NVQs is that a growing number of firms, especially 
large ones, are beginning to use NVQ standards as a tool for restructuring their training and 
management practices with a view to improving the performance of their workforces. At a 
United Distillers plant we visited in Glasgow, we heard how NVQ-like standards played a key 
role in enabling the company to move to a flatter, more efficient management structure 
following a turbulent takeover by Guinness. The company has also used standards to rethink 
what it expects of its production workers and what training it needs to provide to enable them 
to meet these expectations. A recent survey of employers by the Confederation for British 
Industry found that companies that use NVQs in hiring and training employees also tend to 
use ISO 9000 or some other quality management standards in seeking to improve, 
organizational performance, suggesting that the use of these practices goes hand in hand. 

The use of NVQ standards by large companies as a tool for enhancing workforce skills and 
improving organizational performance is generating interest in NVQs among other employers. 
This bodes well for the NVQ initiative, since only if standards add value by answering a real 
business need will they be used by employers in the long run. Moreover, only if they lead to 
real learning by workers will they benefit the economy overall. 

GNVQs: Limited employer involvement in standard-setting 

It was in part a recognition of the limitations of NVQs for providing a sufficiently broad-based 
preparation of young people for entry into the workforce that, in the late 1980s, led to a call 
for the GNVQs. The GNVQs are designed to p,epare for entry into broad occupational 
streams, not to train for specific occupations or jobs. They are aimed primarily at young 
people in the 16-19 age range and are intended to be offered primarily in schools and further 
education colleges rather than in the workplace. This past year they were piloted in 100 
schools and colleges and this year they are being offered in schools and colleges generally, 
though much development work remains to be done. 

GNVQ standards are couched in a format similar to that of the NVQs. Page 1 of Exhibit 8 
contains a list of the units and elements of competence for the GNVQ Level 2 qualification in 
Manufacturing, while the next page details the performance criteria, etc. for the first element 
The broader nature of GNVQ standards compared with the NVQs is evident from a 
comparison of these examples with Exhibits 4-6. In addition, GNVQ programs include 
required units in the "core skills" of communication, application of number and information 
technology, which are designed to be integrated with instruction in the technical subject matter 
units, thereby broadening the GNVQs further. 

Standards for GNVQs are being developed not by the lead bodies but by awarding bodies 
through task forces established for that purpose. By giving responsibility for GNVQ 
development to these task forces rather than to the lead bodies, the government sought to 
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avoid the proliferation of qualifications that has characterized the NVQs. During our meetings 
at the Employment Department we wert told that: "One goal of GNVQ development was to 
limit the number of GNVQ programs to no more than 15 occupational streams. If we'd had 
the lead bodies develop the programs, we'd have 150 or more." Exhibit 7 lists the streams in 
which GNVQs are to be developed, with a timetable for the development of each. 

The task forces charged with setting standards for particular GNVQ streams do however 
include representatives from lead bodies in relevant fields. The role of these lead body 
representatives is to ensure that GNVQ standards reflect competencies required for a range of 
occupations in their fields. The majority of those on these task forces are educators, however, 
which has led at least some employers to express skepticism over whether GNVQs really will 
ultimately help to meet the needs of employers for broadly-trained workers. NCVQ has 
recently set up subject area advisory committees to each of the newly established GNVQs. 
These advisory committees will also include industry representatives whose role will be to 
ensure that GNVQs are implemented and GNVQ standards are updated in line with the needs 
of industry. 

The SCOTVEC National Certificate: Sector Boards for Employer Review of Standards 

Scottish vocational qualifications are based on modules to allow for maximum flexibility in 
creating courses of study appropriate to the needs of the user. Each module is defined by a 
descriptor, which provides all the information a teacher would need to devise an appropriate 
program of study. Exhibit 9 outlines the components of such a descriptor. 

Among other things, the desci tor clearly states the learning outcomes to be demonstrated by 
the student and the criteria to oe used in judging whether the student has indeed achieved the 
outcomes. Thus, the standards are an inherent part of the modules and their development takes 
place in concert with module development. Whoever develops the module for a qualification 
in effect defines the standards for it. However, since every module must be approved by 
SCOTVEC, the awarding body for vocational qualifications in Scotland, SCOTVEC is the 
final arbiter of standards. 

Employers were not much involved in the early development of National Certificate modules 
following the Scottish Action Plan of 1983. Most modules were developed under the auspices 
of the Scottish Technical Education Council (SCOTEC) and the Scottish Business Education 
Council (SCOTBEC), the forerunners of SCOTVEC; they were written by officers of those 
bodies, Her Majesty's Inspectors (the inspection wing of the Scottish Office Education 
Department), and by others selected for that purpose. In recent years, employers have played 
a larger role in developing modules, often in partnership with further education colleges. Also, 
as the NVQ system has come on-line, SCOTVEC has developed modules based on the 
standards developed for the NVQs by the U.K.-wide industry lead bodies. These Scottish 
variants of the NVQs are called, appropriately enough, "SVQs" - Scottish Vocational 
Qualifications. 
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Of more importance for the system of Scottish vocational qualifications, SCOTVEC has 
iastituted a mechanism for review of modules (including the inherent standards) by boards 
representing broad industry sectors. There are eight such boards, each composed of around 25 
representatives from industry and education, with representatives of the latter from both further 
education colleges and universities. These boards are responsible for "validating 1 ' every 
SCOTVEC module in their sector. The actual work is done by a validation panel that 
typically has two representatives from industry, two from education and one from SCOTVEC. 
According to SCOTVEC, the sector boards provide excellent feedback on the quality and 
relevance of its products. 

In addition to validating new modules, the sector boards are instrumental in reviewing and 
suggesting revisions to existing ones. Of the more than 3,000 modules in the Scottish system, 
about 300 undergo such an overhaul each year. By many accounts, the modular nature of the 
Scottish vocational qualifications makes them easy to update. Ian Muir, a training manager at 
GEC Marconi, who is chairman of the Engineering and Manufacture Sector Board, believes 
that the sector boards are effective means of keeping modules (and thus the standards) up-to- 
date. 

With the sector board review, the Scots seem to have found a better way to involve employers 
in the standard-setting process than the approach used with the NVQs, where employers are 
given wide latitude in the setting of standards, and the GNVQs, where employer involvement 
has thus far been limited. As the final approver of modules, SCOTVEC has at least the 
potential to strike a better balance in the setting between the drive of some employers at least 
to train in narrow, job specific skills, on the one hand, and the need of employees (and 
prospective employees) to master broad, transferable skills, on the other. 

This is not to say that finding that balance is easy. Chris Brown of SCOTVEC says that the 
most challenging part of SCOTVECs work is to walk the fine line between over-specifying 
standards (as in "link the blue pipe to the pink pipe to the spotted pipe") and over-generalizing 
them (as in "install devices," where it is not clear whether "devices" refers to a light bulb or an 
engine on the QE2.) 

Some members of our team point to the large number of National Certificate modules as 
evidence that the drive by employers for highly-specialized, occupation-specific standards has 
not been mitigated by the SCOTVEC review system. Yet, while the Scottish system was 
designed to be completely modular, with no formal inter-linkages between modules, there has 
been pressure from the start to offer groups of modules in more cohesive packages. Much of 
this pressure has come from employers, who want to see their employees take a coherent 
program of study rather than just discrete modules. Grouping modules into coherent programs 
has not proved to be a problem. For example, the Scots are creating their own version of the 
GNVQs - the GSVQs - by bundling the appropriate National Certificate modules. 

The need to provide a more cohesive structure to the National Certificate modules was among 
the major recommendations of the Howie Committee (SOED 1992), which was commissioned 
by the Scottish Office Education Department to study the structure of upper-secondary 
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education in Scotland. A precedent for this is provided by the Higher National Certificates 
and Diplomas, which were converted to a modular format following the Action Plan but still 
retain their earlier structure. According to several people we met in Scotland, these have a 
good reputation among employers, who see them as offering a hi^h quality technical level 
education. 

Lessons for the U.S. 

► Establish a single body to oversee the setting of skill standards for all occupations and 
industries. 

Both the Danes and the British recognize the importance of national standards, which allow for 
comparability of programs and portability of credentials (the latter of which facilitates labor 
mobility). In both countries, there is a unitary structure through which standards for 
vocational education and training are set Without a single body or umbrella organization to 
ensure the consistency of standards across programs and industries, it is hard to see how 
standards that are truly "national" could be set. A centralized standard-setting body also 
provides an accessible forum for business influence over the training system. In the U.S., the 
highly decentralized nature of systems for developing standards and curricula for vocational 
education makes it difficult for business to exercise any real control over the content or quality 
of vocational programs offered by schools. 

From this perspective, the National Skills Standards Board that has been proposed as part of 
President Clinton's Goals 2000 initiative seems to make a lot of sense. This board and any 
bodies that underlie it should be constituted to ensure a balance between the primary interest 
of employers in education that is relevant to the work they do, and the interest of employees 
(and educators) for education that also prepares for a lifetime of learning both in and outside 
of the workplace. Denmark, with its coordinated structure of trade committees, each with 
equal representation of business and labor, is in many ways ideal. This structure is not likely 
to be replicated in the U.S., however. The Scottish model, with review of standards by sector 
boards representing industry and education, but with a single awarding body to serve as final 
arbiter, is a model that might be more feasible for the United States. 

Scotland is moving toward a structure for the setting of standards that makes even more sense 
than its current system. In this new scheme, pro^sed by the Howie Committee, SCOTVEC 
would be merged with the body that currently oversees the setting of standards and credentials 
for academic education at the upper secondary and postsecondary levels. This unified 
structure is seen as necessary for promoting integration of academic and vocational learning, 
facilitating cross-over between the different "paths" and raising vocational education to a status 
on a par with academic education. 

► Couch standards in terms of broad goals and frameworks; avoid detailed 

specifications and regulations. Empower those best positioned to affect learning - 
teachers, trainers, administrators and students themselves - to work out the details of 
how to meet the standards. 
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Both the Danes and the British have learned from experience that only by setting standards 
broadly does a system of vocational education based on standards have any hope of keeping 
up with the increasingly fast pace of change in the technology and practice of work. In the 
1960s and 1970s, the Danes set standards for vocational education in great detail. They 
abandoned this approach, having found that it was too inflexible. They have since taken the 
tack of "steering the system by broad targets and frameworks" rather than attempting to control 
it through detailed regulations. 

The English have had more recent experience with the deleterious effects of trying to control 
educational practice with standards in the disastrous start-up of the National Curriculum, an 
effort to establish national standards for secondary school education. The idea received strong 
support from many sectors, including teachers. Evidently, however, the National Curriculum 
Council, which was established by the government to define the standards, was overzealous in 
carrying out its charge. According to a government official we spoke with, "The Council 
over-specified in staggering, patronizing detail what students are to achieve." The result was a 
widespread revolt among teachers, who saw the volumes of standards being produced by the 
Council as a violation of their professional authority. The government has since abolished the 
Council and established a new body, which is more representative of teachers and others closer 
to the practice of teaching. 

The experience of the National Curriculum shows how trying to use standards as a means of 
control frustrates teachers. The opposite approach, using standards to set broad goals for 
student learning, and empowering teachers to figure out how to help students reach those 
goals, has an energizing effect on teachers. Teachers are able to use their professional 
knowledge in finding ways to meet the needs of the indiviaual learner. They are more likely 
to be committed to the system because their role as professionals is valued. Unfortunately, we 
in the U.S. do not show much respect for the professional authority of teachers. 

The approach of using standards to steer rather than control also makes the system more 
responsive to the demands of the market. In the mid-1980s, following the Scottish Action 
Plan, the Scottish Higher National Certificate programs, which had long existed as a more 
applied alternative to the highly academic university preparation system, were converted to a 
competency based format based on broad standards. Since then, enrollment in these programs 
has soared, and they are now favored by employers, who see them as providing solid 
preparation for technician level workers. 

The lessons we can draw as to what methods should be used in standard setting are less 
conclusive. The strength of the Danish frameworks for vocational education is that they are 
stated in terms of what students should be able to do as a result of a particular coarse of study. 
However, they may be too specific about what is to be taught at particular points in a program 
to be useful in the U.S., where there is a strong traditional of "local control" over the 
curriculum. 

The functional analysis methodology being used in the British NVQ system is an intriguing 
alternative to the classical job task analysis approach to defining the skills needed for work. 
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The British experience indicates, however, that the standards methodology itself cannot break 
down the sometimes artificial boundaries between "jobs" to enable standards development 
across broadly defined "work roles." The standards developed through this method will reflect 
how broadly or narrowly work roles are defined. The NVQ standards contain some very good 
models of standards based on broadly defined work roles (see Exhibit 10), but we also saw 
many examples that were clearly grounded in the discrete tasks of narrowly defined jobs. 
Both seem to be the result of a lead body's judgment on how to define an "occupation" that 
merits standards development, prior to actual development of standards. The definition of 
work roles is not always clear-cut, given that similar roles cut across many types of industries 
but may not normally be thought of as related. Interestingly, in the development of GNVQs, 
analysis of labor markets and employers needs prior to the development of standards led to 
very broad grouping of standards around ten "occupational streams." It is still not clear how 
the GNVQ categories will link in practice to the NVQ standards. 

The competency maps that are being produced for the NVQs and GNVQs could be a helpful 
benchmarking tool for the U.S. In the best cases, such maps provide an efficient way to 
represent standards for content and performance in a form useful for- guiding instruction and 
assessment. 

► Understand that standards by themselves will not drive systemic reform. 

Standards, by the very nature of the process through which they are set, are bound to lag 
behind the emerging practices and needs of business. It takes a great deal of time, even in the 
most flexible systems, to develop the consensus needed to establish or change a standard. 
Given the size and diversity of the U.S. compared to the countries we visited, the process is 
bound to be slow even if standards are couched in broad terms. 

The Danish system shows that the standard-setting process in itself doesn't ensure the 
responsiveness of the training system. The main source of innovation and responsiveness is 
the joint development by schools and the partners of actual programs at the local level. This 
on-going collaboration on the design and delivery of programs informs the continual refining 
and adaptation of standards to meet the changing needs of employers. The Danes see the most 
appropriate use of standards as signaling broad national goals that steer the process of 
innovation and as codifying and helping to replicate advances in practice. 

Hence, it is not standards per se> but on-going communication in and among all levels of the 
system about what is expected from the learner that drives innovation in the system. 
Standards can provide a common language and focus for this communication, but they can not 
substitute for it. 



► Begin by building a national consensus about the aims of vocational education. 

In the setting of standards for vocational education, the Danes benefit from the active 
involvement of business and labor based on a partnership built up over the course of a century 
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or more. Lacking a similar history of involvement in education by industry, the British have 
had to aggressively court employers to become involved in the development of the new NVQ 
system. To foster a sense of ownership among emplo>ers for the NVQ qualifications, the 
government turned over the setting of NVQ standards to industry lead bodies. This has 
succeeded in encouraging widespread industry participation in standard-setting, but has also led 
to some serious problems, including the proliferation of standards, many of which reflect 
narrowly-defined, and often low-skill, jobs rather than broad work roles. In hindsight, these 
problems might have been avoided had the British been more selective in specifying the 
groups or families of occupations and the levels of jobs . for which standards should be set. 
Instead, they leapt into the process as though the goal was to set standards for virtually every 
occupation in the labor market, no matter how low-skill or antiquated. 

As the support builds in the U.S. for the establishment of a system of vocational education and 
training based on industry skill standards, it would behoove us to learn from the British 
experience and be more strategic in the development of our system. In particular, the first step 
should be to build a consensus, based on strategic analysis of labor market trends, about the 
purposes the system is to achieve and about the approaches appropriate to meeting those goals. 
Like the British, we in the U.S. will face the challenge of finding a balance between the goals 
of mobilizing broad involvement in standard-setting by industry and generating standards that 
will foster development of the workforce. 

► Use the restructuring of companies around high performance management practices as 
the lever for encouraging employers to use skill standards for training and 
organizational development. 

Active involvement by industry in the setting of standards for vocational education and 
training is essential if the programs based on these standards are to meet the needs of the 
economy for a skilled workforce. The viability of vocational programs where industry is not 
directly involved in setting goals and standards should be seriously questioned. This is the 
case in general with vocational education in the U.S. high schools, where the level of 
employer involvement tends to be low. In Denmark and Britain, vocational education does not 
begin until after compulsory schooling (which ends at age 16) and is offered in institutions 
more akin to our community colleges than our high schools. Like the best of our community 
colleges, these institutions are in constant communication with industry, both through industry 
advisory boards to their programs and through their efforts to provide contract training to 
companies. 

An increasing number of companies in Denmark and Britain are seeing involvement in 
vocational education and investment in the training of their own workers as key elements of i 
larger strategy for meeting the growing challenge of international competition. This larger 
strategy involves restructuring around the ideas promulgated under the labels of "lean 
production," "total quality," or "high performance management." They are reflected in the 
requirements for ISO 9000 certification, which is increasingly seen by companies in Europe 
and throughout the world as a prerequisite to doing business in international markets. 
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In Great Britain in particular, some companies are finding that skill standards are a useful tool 
in becoming high performance workplaces. These companies are using skill standards to 
rethink what they expect of their workers and how they can train and organize workers to 
enable them to meet these expectations. For standards to be used effectively in this way, 
efforts must be made to ensure that they reflect the broad demands of high performance 
work - the high-skill, high-wage jobs of tomorrow - rather than the narrow requirements of 
the jobs of yesterday. A model for achieving this in the U.S. may be found in the Scottish 
approach, where industry has direct input on standards through participation in industry sector 
boards, but where the impulse of small employers especially for job-specific training is 
mitigated by the presence on the boards of educators and by SCOTVEC's commitment to 
ensuring that vocational education is broad enough to allow for portability of learning. 

This approach of setting standards high and then relying on industry, under the force of 
competition, to follow the lead of high performance organizations seems to us to be only way 
to ensure that the process of setting skill standards has a positive impact on the skills of the 
workforce and thus on economic development. The tradeoff is that many companies and 
perhaps whole sectors that rely on low-skill, low-wage work will not take part. Still, this 
seems to be a tradeoff worth making, since the setting of standards for low-skill work is likely 
to use up lots of energy without generating much value in the way of real learning. 

Government can play a role not only by working to keep standards high, but by encouraging 
industry to adopt high performance approaches to management. The British government is 
seeking to encourage employers to use skill standards as a tool for improving human resource 
development through its "Investors in People" program. Run by the Employment Department, 
"Investors in People" is an award given to employers that meet certain criteria of quality in 
their human resource development practices. As opposed to a competition in which only a 
handful of organizations are recognized (e.g., our Malcolm Baldridge Award), Investors in 
People is a certification of high standards of practice that any number of organizations can 
achieve. 

Some of the further education colleges we visited in England and Scotland were consulting 
with local companies on developing training plans and other practices necessary to earn 
Investors in People certification. These colleges had earned the certification themselves. 
Similarly, at least one of the technical colleges we visited in Denmark was working with local 
employers to achieve ISO 9000 certification. The potential benefits for both the college and 
the employer of working together on such efforts are great. 

2. How do standards shape what is taught? 

Main issues of interest to the team: 

How to design standards to promote the development of programs that both 
prepare young people for entry into specific occupations and provide a bioad 
foundation for future employment and lire; 
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How to structure education programs to encourage the transferability of learning. 



Danish Approach 

The history of the modern Danish apprenticeship system has been characterized by an 
increasing role of school in the training cf apprentices. The modern system traces its roots 
back to the Apprenticeship Act of 1889, which made education for apprentices compulsory and 
provided schools with state funding. In the early part of this century, a common school 
curriculum and standard examination requirements were established. The Apprenticeship Act 
of 1956 implemented a decision to gradually move the school part of training from evening 
classes to day classes. A key motivation for increasing the role of schooling has been the 
recognition that, with the increasing pace of technological change, the education of the skilled 
worker has to be broader than that afforded only through training on-the-job. 

The past thirty years have seen continued efforts to broaden the education of apprentices 
further, with a focus on the school part of training. The 1991 reform codified as broad goals 
for the system the results of experimentation and debate over a decade or more on how to 
accomplish this. The reform brought about major changes to the first year of instruction in 
school, but the overall structure of a course of study in vocational education, in particular, the 
"sandwich model," where students alternate between periods of training in-school and on-the- 
job, remained as it was before. The current structure is shown in Exhibit 12. 

The impetus for the reform came from two directions. On the one hand, the partners (see 
p. 7), through the trade committees, pushed for broader education out of a recognition that not 
only was the pace of economic and technological change increasing, but, for this reason, 
workers were changing jobs much more frequently. On the other hand, educators and the 
Ministry of Education sought to' enhance the flexibility of students in vocational education to 
cross over to an academic path, and, of most importance, to continue on at some point in 
higher education. 

In relating the reasons that the partners were behind the recent reform, J0rgen Andersen of the 
Danish Metalworkers Union told us: 

"The apprentice thinks he will have the same job for which he is being trained 
all his life. That is a fantasy. The key is learning to learn. This key element 
must be part of all education .... The trade committees have pushed hard for 
providing as broad an education as possible, even though a narrow education 
would be less costly. This is based on the realization that a [broader] 
education gives one greater flexibility to change jobs." 
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We heard a similar view from Finn Larsen of the Danish Labor Organization. 

"Behind the reform was an effort to change people's thinking: no longer 
will a union card alone be a guarantee of n job; it is the ability to learn 
that will guarantee you employment in the economy of the future." 

To accomplish this broadening, the partners consolidated programs in particular trades, 
reducing the number from 270 or so to under 90. The partners also advocated articulating as 
one of the main goals of the reform the idea that "education should be holistic." By this, they 
sough* to find a better balance in vocational education between the theoretical and the 
practical. Certain trades also saw this as encouraging the teaching of skills not traditionally 
emphasized in the academic canon. Exhibit 13 contains a list of such skills. The list was 
developed by the Metalworkers Union through a 1986 project in which focus groups of 70-80 
of the leading employers in the metal working industry were convened and asked "What do you 
foresee will be the 'basics* required of workers in the 1990s?". (We gather that these skills 
are reflected in the frameworks for the metalworking trades, but these frameworks were not 
available in English.) 

Holistic education was also advocated by Danish educators, but to them this meant promoting 
interdisciplinary instruction within the academic domains and discouraging the teaching of 
subjects separately - "because the real world is not broken down in that way." At the same 
time, the educators' answer to the need for a broader education in order to enhance student 
flexibility was mainly a push for more academic content in the curriculum. The 1991 reform 
increased the number and rigor of required academic courses and added an examination to be 
taken after the subject matter courses are completed. 

The increased academic requirements have made it difficult to realize in practice the goal of 
making vocational education more holistic, at least in the sense intended by the partners. 
Academic and practical instruction remain separate, especially in the first year. The required 
academic courses are given mostly in the first year so that students can get the exam out of the 
way and proceed to practical training in subsequent years. According to the practical teachers 
we spoke with, this cuts into the time students can spend in hands-on learning in the workshop 
during the first year. Some of these teachers also believe that the heavy load of academic 
courses in the first year discourages students and leads to a high drop-out rate. In both the 
technical colleges we visited, over 20 percent of students drop out in the first 20 weeks. Many 
of these drop-outs enter vocational education in another field, but at least some are 
discouraged and never continue their education beyond that point. 

The separation between academic and practical instruction is difficult to bridge because the 
two are governed by different standards and taught by different teachers. While the trade 
committee frameworks specify the structure of the entire program, it is left to educators, 
through the Ministry of Education, to set standards for the academic subject matter courses. 
Teachers of academic courses are required to have university degrees, whereas teachers of 
practical subjects are not. While some of the latter do have degrees, all have come through 
the vocational education system* In practice, there is some communication between academic 
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and practical teachers, but not much, according to the teachers we spoke with. By insisting 
that education be holistic, the 1991 reform obligated academic and practical teachers to work 
more closely together, but failed to remove the structural barriers to this sort of cooperation. 

This structural divide between the academic and the practical is not easy to overcome, in part 
because it reflects differing views of how to achieve portability or transferability of learning. 
When we asked a practical teacher the reason the reform called for such a heavy academic 
course load in the first year of the program, he responded: 

"Academics at the University of Alborg were behind this. They did a study 
that found that people in the workforce are changing jobs more. They said, 
'Aha! Then we should put greater emphasis on academics in vocational 
programs, because that will lead to transfer of skills/ 1 ' 

When we asked this same teacher what he thought was the best way to prepare young people 
for a lifetime of frequent job changes, he said, "give them lots of experience working with 
tools." A similar view was expressed when we asked representatives of the partners how they 
can justify encouraging young people to become apprentices, when Iwo-thirds or more cannot 
find jobs in the trades in which they were trained. 

"If you have a skill, say, you are a barber, then somehow you will be able to 
use your skill, even if not in the field in which you were trained." 

"[The inability of apprentices to get jobs in their trade] is not a problem 
because you can't be disciplined at anything if you are not disciplined in at 
least one thing." 

By this view, it is the experience of having mastered a set of skills, not abstract knowledge, 
that enables one to leam new things and deal with unfamiliar situations. Some of the practical 
teachers we met also emphasized the importance of the sense of accomplishment and 
confidence that comes from mastering a skill, producing a product, or solving a practical 
problem. Most students do not get this from book learning alone. This is overlooked in the 
current model, where students spend much of the first year taking academic courses like those 
they took in Folkeskole. 

Practical learning can motivate students to want to acquire a more theoretical understanding, 
according to one teacher. 

"The students we get are tne weak students from the Folkeskole, Many of 
them are immigrants. We start them out with the same school stuff they took 
in the Folkeskole and they hate it. They feel like fa^ures. But, later in the 
program, when we are working on projects in the workshop, students are very 
eager to do the academic work." 



Center for Learning and Competitiveness 

The Role of Standards, Assessment and Credent ialing 



Page 25 



A better approach, according to this teacher, would be to start students off with a set of hands- 
on projects that get them interested in learning and then to build-in theoretical instruction, as 
needed, to understand what they are doing in the practical work. There would still be plenty 
of time for specialization," the teacher pointed out. 

This is closer to the way learning is organized in the "practical period" of training following 
the first year. Ideally during that period, students work on projects with coaching from the 
teacher, who provides instruction or points to other resources when questions of theory arise in 
the course of carrying out a particular project. The 1991 reform has encouraged this approach 
of "theoretical learning as needed to support practical learning" in the practical period. At the 
Metal Industries Technical College outside Copenhagen, classrooms used to be separate from 
the workshop. The school is now being redesigned so that classrooms encircle the workshop. 
Teachers and students will use the classrooms for planning projects to be carried out in the 
workplace and for instruction on topics relevant to projects underway there. This 
reorganization of the school mirrors the restructuring in at least four of the companies we 
visited in Europe, where R&D operations were being moved next to the factory in order to 
facilitate symbiosis of theory and practice. 

Despite some advances in the direction of more holistic education, the Danes are still 
struggling with how to accomplish this. A key challenge will be to break down the separation 
of academic and practical learning that characterizes the first year of technical programs. This 
will require academic and practical teachers to work together and to understand and respect 
each other's cultures. Some of the efforts to change teaching in this direction are discussed in 
the section on teaching methods. 

British Approach 

Like the Danes, the British have been trying to come up with a design for vocational education 
that is broad enough to promote transferability of learning but is not so broad that students fail 
to acquire skills that help them to get jobs. The most recent attempt is the General National 
Vocational Qualifications (GNVQs), which, as mentioned above, are designed to provide a 
framework for school-based vocational education programs, primarily for students of ages 16- 
19. They are intended to signify preparedness for entry into a broad stream of occupations, 
not competence in a particular occupation (the latter being the role of the MVQs). 

The GNVQs were developed to meet two major needs. The first was the need to create a 
school-based alternative to the work-based NVQ, which, unlike the NVQ, would aim to teach 
broader skills than those specific to a particular occupation. The second was to provide a more 
applied alternative to the highly academic and highly exclusionary A-level examination system 
as a means of preparation for university-level study. 

GNVQs are made up of a series of units that are defined by competence maps similar in 
format to those used to define NVQ standards. (See Exhibit 8 for an example of a GNVQ 
competency map.) As with the NVQs, each GNVQ unit is competency-based, which means 
that a student completes a unit by demonstrating the competencies required by the unit. 
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Students document their progress toward meeting the various competencies in a portfolio of 
evidence. Like NVQs also, GNVQs are defined in a framework of five levels. Exhibit 14 
shows the structure of GNVQ Level 3 programs. 

GNVQs differ from NVQs in that standards of competence are defined more broadly. 
Moreover, GNVQs modules include core skills units in addition to content area units. The 
core skills units represent areas of competence that cut across technical or content-area skills 
and knowledge. They are seen as fundamental to learning new knowledge and skills; therefore 
their inclusion in the GNVQs is a key strategy for ensuring the transferability of the learning 
that takes place in GNVQ instruction. The core skills units are required to be taught in 
conjunction with the content area units. The challenges posed by this requirement are 
discussed in the next section. 

The core skills are as follows: 

• Communication 

• Application of Number (Numeracy) 

• Information Technology 

• Personal Skills (Working with Others and Improving One's Own Learning and 
Performance) 

• Problem-solving 

• Modem Foreign Language 

• Personal Effectiveness (Planning, Managing Others, Managing Oneself and 
Using Intellect) 

Under the current scheme approved by the government, only the first three core skills units are 
required; the fourth is highly recommended. The elements of competence for the first three 
required core skills are outlined in Exhibit 15. Problem-solving is not required because it is 
thought to be too dependent upon the context of learning, and thus not amenable to 
standardization and reliable assessment, as are the first four. Standards have been defined for 
Modern Foreign Language units, but the government chose not to include them among the 
required units because it was concerned that it could not field a sufficient number of foreign 
language teachers. Finally, the four personal effectiveness dimensions are not required because 
they are considered to be too subjective and dependent on organizational context and culture to 
allow for standardization. Still, the government thinks that they are important and wants to 
promote their use. Using models developed by large corporations, the government is working 
on frameworks for measuring, documenting and tracking these dimensions that are more 
accessible to schools and small organizations. 

Development of core skills units has been in progress for at least a decade, much of it through 
government-funded demonstration projects. A well-structured program of research has been 
designed around the piloting of the GNVQs over the past two to three years. (See Oates 
1992 for an excellent review of this research.) This research will continue now that the 
GNVQs are being offered on a national basis. 
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Much of the early work on core skills was done in Scotland, in response to the Scottish Action 
Plan, which saw broad enabling skills as key to ensuring adaptability in employment and 
further learning. The core skills are clearly reflected in National Curriculum modules, both in 
separate core skills modules (e.g., communication, application of number, etc.) and as 
outcomes embedded in vocational modules. 

Despite more than a decade of experimentation, the use of core skills in teaching in Scottish 
vocational education is far from universal. As Sheila Clarke reports in a recent review of 
vocational education in Scotland, 

[I]n evaluating the implementation of the NC, an HMI report concluded that 
while examples were found of programmes with strong emphasis on core 
skills, sucn good practice was attributable to effective teaching or programme 
design by individuals rather than to college strategy. (Clarke 1992: 27) 

Clarke cites a number of reasons for this, including "the lack of models of good practice and 
staff development," and "uncertainty over whether there is transferability of core skills." 

Use of core skills is not more widespread in part because they are not required to be part of 
National Certificate programs. As was indicated earlier, the National Certificate modules were 
designed to allow users maximum flexibility in putting together NC modules to meet their own 
needs. Giving users this sort of license has its advantages, to be true, but the result is not 
always in the be*t interest of the learner. Clarke cites other research suggesting another reason 
core skills are not more widely applied: small and medium sized companies have not adopted 
core skills initiatives advocated by larger firms, but see their training needs as more 
immediate. Therefore, colleges have left out core skills from the programs they design for 
these companies. The result is narrow training for the student rather than an education. 

As was mentioned above, the wide latitude that the National Certificate system gives the 
learner to assemble a program of study from modules of their choice is currently under 
scrutiny. Indeed, many programs are offered as group awards, more structured groups of 
modules that in many cases include core skills modules. And SCOTVEC has developed 
Scottish equivalents of the GNVQs « the GSVQs » by packaging existing National Certificate 
modules, including those for the core skills. 

What we find attractive about the GNVQs is that the core skills are a required part of the 
framework and, moreover, that they are required to be integrated with the teaching of technical 
or content area subject matter. It is still too early to tell the extent to which this integration 
will promote transferability of learning. This is a key item on the research agenda for British 
vocational education. In the U.S. we should keep an eye out for results of this research, but 
we cannot afford to wait until they are in to begin similar work of our own. 
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Lessons for the U.S. 



► Ensure that goals and standards for the system reflect a balance between theoretical 
and practical learning 

The balance needs to be reflected in the standards if it is to be manifested in teaching and 
learning. One reason that the Danes have had such problems with achieving more holistic 
education in the first year of vocational education programs is that the standards that govern 
the academic courses are established independently of those for the practical studies. In 
Scotland, there are discrete core skills modules and core skills are reflected in the outcomes for 
vocational modules, but core skills are not universally reflected in teaching because they are 
not required elements of National Certificate programs. 

The GNVQs present a model for achieving the kind of balance that should be reflected in 
content standards for vocational education in the U.S. Core skills are defined with their own 
standards separate from those for subject matter domains, yet the GNVQ frameworics require 
that core skills be taught in conjunction with subject area content. 

► Question the assumption that academic learning leads to superior transferability of 
skills and knowledge. 

Both the Danes and the British are trying to create vocational programs that are 
occupationally-relevant and yet that allow for transferability of skills. Neither has found the 
definitive answer about what makes for transferable learning. Still, both are discovering - 
spurred by the work on training and organizational learning by high performance companies — 
that the answer lies in a better balance between theoretical and practical learning. This should 
cause us to question the assumption, underlying the design of U.S. education at all levels, that 
academic learning leads to superior transferability of skills and knowledge. 

In the best examples from Denmaik and Great Britain, learning is organized according to 
projects designed to allow students to tackle practical problems in-depth and motivate them to 
investigate theoretical issues as needed. This is the case in the Metalindustries Technical 
College in Denmark, where students undertake practical projects ir the workshop with 
coaching from the teacher, using the classrooms that ring the workshop as needed to explore 
theoretical issues relevant to their project work. 

The balance between academic and applied learning is reflected in the teaching of academic 
subjects in Denmark. This is evident from three pages in a Danish mathematics textbook for 
the ninth grade, presented as Exhibit 16. The discussion of geometry begins with pictures 
with geometric manipulatives that the student could actually make and play with in order to 
get a hands-on understanding of geometric fornis. On the same page, the connection is made 
to how geometric forms are reflected in real world objects - in this case a bridge - and to the 
use of geometric skills -- in this case in the construction of a frame for a house. On the next 
page, further connections are made to the geometry of angles in the game of pool (a common 
Danish pastime). Only after the real-world links have been established are the abstract 
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notations for geometric properties of angles and the like introduced. Some students may never 
use this last form, but at least a connection has been made to application. 

In our system, the connection is not made, since we only teach the abstract math. Those 
students who go on to do advanced mathematics will lack a grounding in hands-on experience 
and understanding of the practical application of mathematics. Students who have difficulty 
with the abstract concepts are given "applied mathematics," but only after they have developed 
a sense that they are no good at mathematics. 

3. What is the role of standards in promoting innovation in teaching? 
Main issue of interest to the team: 

• How to use standards to encourage methods of teaching that engage students in 
learning and enable them to apply what they learn to new problems and 
situations? 

Danish Approach 

According to Jens Pehrsen of the Danish Ministry of Education, the goal set forth in the 1991 
reform of making vocational education "holistic" carried with it a new set of obligations for 
the teachers of the system. To achieve this goal, says Pehrsen, 

• Practical teachers and academic teachers must work together, 

• Partners have to say to practical teachers and skilled trades people (who serve as 
mentors and assessors for apprentices) that young people need to have a broader 
education than they had; 

• Teachers' unions must stress to the academic teachers the need to make education 
more interdisciplinary and applied. 

Two years later, the teachers of the system are still struggling with how to fulfill the new roles 
called for by the reform. This is not to say that all this is new. The ideas promulgated by the 
reform have all been developed and tried out in demonstration projects or by teachers and 
colleges working on their own. Now the challenge is to spread these ideas across the system 
so that innovations become common practice. 

Two innovations around which there seems to be a lot of interest and experimentation in 
Denmark are team teaching and project teaching. 

Team Teaching 

One of the thrusts of the reform was to take teaching out of the subject and put it into the 
team. Within Danish vocational education, according to Jens Pehrsen of the Ministry of 
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Education, some have long argued that students should not be taught subjects separately, 
"since the world that they are being prepared for is not broken down that way." For at least a 
decade leading up to the reform, the Ministry funded R&D projects aimed at creating models 
of interdisciplinary teaching. In these projects, teachers served as the primary "researchers." 

Beyond these demonstration projects, teaching has still tended to be along disciplinary lines. 
However, team teaching seems to have long been the norm in the operations through which 
the technical colleges offer contract training to companies. Jens Erik Simonsen of Arhus 
Technical College said that the short time frame in which most contract training has to be 
developed typically requires that they be designed by an interdisciplinary team. Still, he says 
that this approach has not spilled over much into the planned courses of the regular 
curriculum. 

The biggest challenge is to get practical teachers and theoretical teachers to work together as a 
team. At the Metalindustries Technical College on the outskirts of Copenhagen, teachers are 
working in teams to design and teach the program for an entire level of instruction, for . 
example, the entire first year program. These efforts mirror those of high performance 
companies that assign to an interdisciplina^ team responsibility for developing and managing 
an entire segment of an operation, for example the production line for a given product. This 
work has been going on for about a year, according to Jonna Blak of the college. The results 
have been good, with academic and practical teachers working together for the first time. 
Scheduling of students and classroom has become much more complicated, so the college has 
hired a scheduling coordinator. 

Project Teaching 

Another innovation in teaching encouraged by the reform is teaching through projects rather 
than lectures and lessons. Project teaching is often linked to team teaching since projects can 
often be developed and managed most feasibly by a team of teachers. 

With project teaching as well, there has been a good deal of experimentation over the past 
decade in Denmark. An especially impressive example was the "Metalworking for the 90s" 
program, an experimental training program conceived by the Danish Metalworkers' Union and 
run jointly with industry through the metalworking trade committee. The purpose of the 
program was to explore new approaches to educating skilled workers in the face of dramatic 
changes in the metal working industries. The program began in 1985, with the effort, 
mentioned earlier, of the metalworkers* trade association to define the skills that would be 
needed by skilled workers in the metal industries in the 1990s. (The list of competencies 
identified are shown in Exhibit 13.) 

Apprentices in the program were given intensive training in problem-solving and teamwork. 
They were then required to carry out a series of projects, some of them several months in 
duration, that required them to work as a team. The instructors served as coaches, monitoring 
each student's performance through frequent meetings. The students' performance in these 
projects was used as the main criterion for deciding whether or not they passed the 
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journeyman's exam. (Apprentices in this experimental program received an equivalent amount 
of technical training as those in the regular program since an additional year was required by 
the experimental program.) When the program ended in 1992, over 75 apprentices had been 
trained in this way. 

The program proved to be quite challenging for the teachers involved, according to J0rgen 
Andersen of the Metalworkers' Unioa For one, the teachers found that they lacked some of 
the technical skills required to carry out the project. They had to pick these up just as the 
students did. Also, they had a hard time at first figuring out how to create a learning 
environment in the school that mirrored the workplace closely enough that the students would 
"get into it, could feel that they were actually working in an actual shipyard or some such." 
Most problematic, according to Andersen, was that the teachers did not know how to plan and 
manage a project. This way of operating was foreign to most of them. Even though courses 
were provided for teachers in the program, Andersen admits that they were not given enough 
support of the kind they needed. For one, the teachers would have benefited from training and 
support around project management, he believes. 

Andersen and his colleagues have embarked on a new venture aimed at promoting teaching 
projects developed and carried out jointly by technical colleges, gymnasiums and employers. 
In one project, participants collaborated on the construction of a weather station. Four 
metalworking apprentices were involved. Andersen boasts that "they are much better at 
planning and organizing the project than the gymnasium students and the gymnasium 
students know it." 

The move to project-based teaching is getting some encouragement from the budget cuts that 
colleges have been forced to make due to systematic cuts in funding from the government over 
the past couple of years. One response to the mandate to do less with more, according to 
Jonna Blak, is what is called "teacher-less teaching," where the teacher gives the students a 
project and makes themselves available to the students if they need help. That way, the 
teacher can manage 2 or 3 classes at a time. The alternative is to add more students per class. 
With this approach, students tend to help each other out, so the students are carrying some of 
the load of teaching. 

Others indicated, however, that project teaching may be more expensive than the traditional 
lecture approach to teaching, precisely because you cannot simply pack more students in a 
classroom. Referring to the members of Parliament who are advocating the push for more 
holistic education, an official in the Ministry of Education told us, "they do not understand 
how expensive project teaching is." Project teaching requires a great deal of preparation time 
on the part of the teacher. Also, the need to be available to students to provide support and 
monitoring of progress does not permit the kind of student-to-teacher ratio possible in the 
traditional classroom. The solution to the funding dilemma lies in a new way of thinking, 
according to one administrator we spoke with: 

"The expense of project teaching is not so bad as it seems, but you must not think about one 
instructor for so many students. You have to think of a team of instructors managing several 
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projects, each with a small group of students. You also have to be willing to give students 
more responsibility for the work." 

How do teachers feel about ali this? 

The administrators we spoke to had mainly good things to say about the impact of the reforms 
on teachers and teaching. 

"Teachers have found that they like their fellow teachers better. Now 
academic and technical teachers talk when they did not in the past. Their 
education credentials matter less. Under the new system, it matters more what 
you can do. M 

"There are problems, sure, but this will work in the end because teachers and 
students arc enjoying what they are doing and you have to enjoy what you arc 
learning to be motivated." 

The teachers we met tended to be more reserved in their judgments. One teacher said that she 
liked the notion of projects, but she did not have time to work on them. Communication 
among the teachers involved in designing projects was difficult, even though time was set 
aside for planning. Also, the subject-matter exam that students were required to take at the 
end of the first year was not relevant to what she and her colleagues wanted to teach through 
the project. (The exam is discussed below in the section on "Assessment.") Another teacher 
said that teaching through projects "demands more of you, but it enables you to see students as 
they develop." 

British Approach 

In Britain, as in Denmark, the goals and frameworks articulated by the reforms in vocational 
education arc causing teachers to rethink how they teach. Like Denmaric, Britain is benefiting 
from a decade or more of experimentation with new methods of teaching, although the British 
too are a long way from truly systemic reform. 

The GNVQ framework is promoting team teaching by teachers in different disciplines. 
Because the core skills arc designed to be integrated with the technical subject matter, there is 
pressure on colleges and teachers not to offer separate modules in core skills. According to 
the staff we met with at the Employment Department (ED), when the GNVQs were being 
developed, there was pressure from some quarters not to require an integration of core skills 
units with vocational units in the GNVQs. The main reason was that teachers have not 
traditionally taught in an integrated way, and it was proposed to provide support for this more 
gradually. The Employment Department disagreed, on the principle that if this is not 
confronted head on, things will never change. The ED was resolved that things might not go 
well in the first pilots, but at least it would be a start. Plus, many difficulties would be 
worked out at the level of the school or college. 
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Based on our brief visits to colleges and schools in England, this strategy seems to have 
worked. At Monkwearmouth College in Sunderland, which was one of the 100 schools and 
colleges that took part in the pilot of GNVQs this past summer, efforts are underway to 
integrate core skills into technical modules, but, in the words of one school official, "we have 
a long way to go." 

During last year's pilot, a number of lecturers at Monkwearmouth expressed unease at having 
to incorporate core skills in the teaching of their technical domain. In many such cases, the 
lecturer did not feel qualified to teach a particular core skill. Application of number and use 
of technology were commonly mentioned. The college found that using teams to develop and 
deliver modules helped to alleviate these anxieties while also accomplishing the required 
integration. Still, the college believes that not all core skills lend themselves to integration. It 
has established "flexible learning workshops" to provide instruction in those core skills 
elements that can not easily be integrated into the vocational units. 

Team teaching is more common in the further education colleges of Scotland, thanks to the 
Action Plan of 1983. One teacher who has been involved with the colleges for a long time 
told us that, before the Action Plan, lecturers mostly worked alone, producing their own 
teaching materials and tests. The Action Plan called for curriculum to be produced in modules 
and required that modules be subject to "moderation," that is, that they be designed according 
to approved procedures and, more importantly, that they reflect the same standards across 
centers where they would be offered. The need to moderate modules encouraged developers 
to work in small teams. The use of teams spread with the rapid development of National 
Curriculum modules. Teams were traditionally used in teaching of the Higher National 
Certificate programs, so when the HNCs were converted to a modular format, the use of teams 
was well-received by teachers in those programs. 

Several people indicated to us that the modular nature of the Scottish vocation encouraged the 
use of projects in teaching and assessment. During our visit to Cardonald College in Glasgow, 
two lecturers described to us projects that spanned several modules. Others we spoke to 
predicted that the required integration of core skills units with vocational units in the 
GSVQs - the Scottish version of the GNVQ - would increase the use of project teaching in 
those progiams. 

One obstacle to the use of projects for teaching in schools and colleges is the difficulty 
educators have in giving up coverage of the curriculum to allow for in-depth exploration of the 
sort afforded by project learning. This concern was raised by a training manager at a 
multinational in Scotland who has worked to promote the use of projects for teaching and 
learning in schools at the compulsory, further education and university levels. 

"What you hear from schools is, 'How can we take time out to organize 
projects when we have to cover all these other things?' I say that having a 
curriculum is one thing, but what we've got to do is prepare young people for 
the world of work. Industry today needs people with the 'how to skills' not 
the 'what skills/ Schools teach what they know best: facts and knowledge, 
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not thinking and problem-solving. Projects are excellent experience for 
students because they give them an opportunity to come up against new 
problems and circumstances. They build students' confidence to try new 
things, and what we sorely need in business is people who are willing to take 
chances. To make a project work t youVe got to lay down the goals and then 
let things happen. But this takes management skills that the school people 
don't have. Plus, schools are not accustomed to giving control to students." 

He went on to say that there is a similar reluctance on the part of business to set a strategy 
and then empower employees to figure out how to achieve it. But business needs to change, 
he believes, or it will not be able to keep up with the demands of fickle markets. 

Lessons for the U.S. 

> Rely on standards for what students should be able to do, rather than directives of 
how teachers should teach, to drive innovation in teaching. 

The experimentation with methods of teaching that we witnessed in both Denmark and Great 
Britain has been catalyzed not by new directives on teaching, but by changes in goals and 
standards for student learning. Danish teachers are grappling with ways to achieve the goal, 
set forth in the recent reform, of making the education of students more holistic. Scottish 
educators are finding innovative ways to enable students to achieve the outcomes specified in 
the National Certificate modules - including the use of long-term projects that span several 
modules. The required integration of core skills units with vocational units in the GNVQ 
framework seems to have great potential for encouraging interdisciplinary teaching. 

Both countries are finding that competency-based standards - that is, standards defined in 
terms of what students should be able to do - tend to promote instruction that is 
interdisciplinary and project-based. Competency-based standards also allow for more 
flexibility in where and when learning takes place. Such standards that are explicitly based on 
what students will be expected to do in the world outside of school encourage the 
demonstration of competencies either in the context of real work applications or practical 
projects in school. 

In both countries, unlike the U.S., there is a strong tradition of teachers as professionals, 
qualified to decide how best to enable students to achieve standards of competence. Under the 
on-going reforms in these countries, some excellent guidance materials have been produced by 
government agencies and awarding bodies. But in general, the strategy has been to let schools 
and teachers figure out how to achieve the goals themselves, on the principle that the many 
problems that will arise are best worked out "in the field," rather than in a central office. 

This strategy will not bring about changes quickly. The Scots have been experimenting for 
over a decade with using interdisciplinary, project-based instruction to help students achieve 
competency-based standards, and still these practices are by no means universal in Scotland. 
While this approach makes for slow change, it may well result in more lasting change, since 
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teachers will convince themselves, not be told by others, what works in helping students to 
learn better. 

► Provide training and support to teachers exploring new ways to help students achieve 
the standards. 

If the experience of the Danes and British is any guide, much of the support needed will be in 
the way of administrative support at the school level, for example, to rearrange schedules to 
accommodate project teaching and free time and space for teacher teams to meet. 

Teachers will also need extensive, on-going training. The experience of the Danish 
Metalworkers' "Metalworking in the 90's" program suggests that teachers would benefit from 
training in techniques for project planning and management. 

► Dorit let the urge to "cover the material" get in the way of rich learning experiences. 

An articulate case for this position was made by the training manager quoted above, who 
argued that schools should be more concerned about what students are able to do than what 
they know and, therefore, should encourage projects and other approaches to teaching that help 
students develop process skills, not just content knowledge. 
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ASSESSMENT 



1. How is the performance of learners assessed and documented? 

Main issues of interest to the team: 

• How to use assessment not only to measure learning but also to motivate it; 

• How best to measure and document the progress of the learner over time. 
Danish Approach 

The Danes assess the performance of students in vocational programs through a combination 
of on-demand and embedded assessment. The approach to assessment differs markedly 
between the first year of vocational education and the last two years, where assessment follows 
the more traditional approach. This difference stems from the fact that standards for the 
academic part of vocational education, which is concentrated in the first year of a student's 
training, are separate from those for the practical part, which begins in earnest in year two of 
most vocational programs. 

Assessment in Year 1: Culminating Academic Exam 

The push in the 1991 reform for greater academic rigor resulted in more required courses in 
mathematics, science and languages and a culminating exam of academic subject matter that is 
taken after the required courses are completed. Most students take the exam at the end of 
their first year, so they can devote full time to practical learning during the succeeding years. 
Some of the vocational teachers we spoke with complained that this new arrangement cuts into 
their ability to do much in-depth practical learning in the first year, not only because of the 
greater load of academic courses students must now carry, but, in the words of one instructor, 
"the students start cramming for their exams in April; after that it is impossible to get their 
attention." 

The exam consists of a written and an oral part. The written test is developed by the Ministry 
of Education, which commissions teachers from around the country to write the questions. All 
students in the same family of occupations take the same test. The tests are secure, so that 
even the teachers are not permitted to see the questions in advance. The teacher leaves the 
room during the exam. Afterward, the teacher makes a copy of the exams and sends the 
originals to another college for grading by a teacher there. One of our team members pointed 
out that, perhaps in a small way, this practice contributes to "norming" of performance 
standards, since it enables the teacher doing the scoring to see the quality of work done by 
students of another teacher. 
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The oral test is developed locally. It is carried out by the teacher and a colleague from 
another school. Students pick questions prepared by the teachers out of a hat and have twenty 
minutes or so to prepare their response. 

We got conflicting messages about how important these exams are. Some of the students we 
spoke with seemed concerned that if they failed the tests they would be dropped from the 
program. Teachers and administrators indicated that the exams do not carry such high stakes. 
Certain of the companies we visited do look at how well a student has done in school in 
Danish and mathematics especially; but they placed much more weight in making selection 
decisions on the interests and aspirations of the applicants. The primary purpose of the written 
exam especially seems to be to motivate students to take their first-year academic courses 
seriously. 

Assessment in Practical Training: Continuous Assessment of Competence 

Assessment during the "sandwich" phase of the programs, when students do alternating stints 
in the school and the workplace, is much more embedded in the daily teaching and learning 
process. As one team member observed: 

"Assessment is continual and so built into the process and so tailored to the 
way an individual student communicates his or her competence. These kids 
have so many ways to demonstrate that they can do this work and that they 
can achieve." 

An important tool for monitoring the apprentice's progress during this period is a log book 
that the apprentice takes back and forth between school and workplace. In this log, the teacher 
and workplace mentor record the major tasks that the apprentice has undertaken toward 
meeting the standard and rate the apprentice's performance on each task. They also make 
general comments concerning the apprentice's behavior and indicate any need for follow-up. 
A mentor at Radiometer told us that "if a teacher thinks that a student needs work on a 
particular skill, he will indicate that in the student's log and we will provide additional 
instruction as necessary." Exhibit 17 is a sample page from such a log. i \ the end of each 
school stay, the apprentice's teachers fill out an evaluation in which they i ssess the student's 
progress during the period and evaluate the success of their teaching in helping the student 
make this progress. A sample of such an evaluation form is given in Exhibit 18. The log 
allows the teacher in the school and the mentor in the workplace to keep close track not only 
of the apprentice's progress, but of what the other is doing. The best teacher or mentor will 
adapt what he or she is teaching to reinforce what the student is learning in the other's 
domain. 

Teachers and mentors also communicate directly with one another as necessary about an 
apprentice's progress. The several mentors we asked said that they are in frequent contact 
with teachers at their apprentices' technical colleges. Most added that they would have no 
hesitation contacting a teacher were an apprentice having difficulties. 
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Some companies do their own periodic evaluations of their apprentices' progress. One 
company we visited, Danfoss A/S, a manufacturer of metal valves and other specialty 
products, evaluates its apprentices every three months. The evaluation is carried out by the 
apprentice's sponsor (i.e., mentor), who assesses the apprentice on the following dimensions: 

• Professional (i.e. technical) skills 
Ability to learn 

• Methodical and careful 

• Well roundedness 

• Cooperative 

• Enterprising and independent 

• Productive. 

Another company we visited, Radiometer A/S, a manufacturer of medical testing equipment 
with 4,000 employees (making it a fairly large company for Denmark), evaluates its 
apprentices along similar lines using the sophisticated performance evaluation system it uses 
with all of its employees. The Radiometer system was particularly impressive in that it ties 
the skill development plans for each employee to the strategic business plan for the company. 

Companies such as these take seriously the assessment of "soft skills," which are seen as every 
bit as important as technical skills and knowledge. This view is reflected in a comment by a 
mentor at Radiometer. 

"A youth apprentice can fail on social behavior. If he cannot work as a 
member of a team, he will not succeed in today's workplace and therefore will 
not have what it takes to be a journeyman." 

The Journeyman's Examination 

The culmination of an apprentice's training is the journeyman's examination. During the final 
days of the last school period, the apprentice is asked to undertake a project. An apprentice in 
machining, for example, might be asked to produce a "master piece." An apprentice in 
electronics might be asked to troubleshoot a system into which faults have been introduced. 

On the last day, two outside examiners come to the school to evaluate the apprentice's work. 
The examiners always consist of one representative from labor and one from industry. 
Examiners are chosen by the trade committees according to set rules designed to avoid 
patronage. The 1991 reform law now requires that examiners undergo training to ensure 
reliability of judgments. They are required to meet ahead of time to review the assignment the 
apprentice has been given and agree on the criteria for judging how well the apprentice has 
done. 

The examiners follow an agreed-upon script in asking the apprentice about their work. We 
were told that examiners are as much concerned about the process by which the apprentice 
carried out the assigned task as with the quality of the final product, and so will ask the 
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apprentice to reflect on how the work was done. J0rgen Andersen of the Metalworkers Union 
said that while older examiners tend to focus on technique, younger ones will base their 
evaluation on "soft skills" as well - how well did the apprentice present his work, how 
articulate was he in answering questions, how good a job of planning and trouble-shooting did 
he do? This is being encouraged by the trade committees and the recent reform law. The 
teacher is present during the examination, and may take part in questioning the apprentice, but 
the final decision rests with the outside examiners. 

The 1991 reform law mandated that the journeyman's examination include a theoretical as well 
as a practical part. The apprentice's teachers write the questions for the theoretical part, 
usually on a topic different from the one covered in the practical exam. 

Only about two to three percent actually fail the journeyman's test. In the rare instance that 
an apprentice does fail, the company, not the school, bears the brunt, since it must bring the 
apprentice on at a full journeyman's wages and provide tutoring and support to prepare the 
apprentice to retake the exam. No one fails on the second try. 

The journeyman's exam is not a make or break test There are no surprises on the exam. By 
the time the apprentice takes the exam, he or she has done similar work many times before, 
and is thoroughly familiar with the tools and systems he or she must deal with on the exam. 
We spoke with apprentices who were preparing for their examinations in electronic mechanics 
(machine control). They did not seem to be overly anxious about the impending exam. In 
fact, they said that they were much more concerned about getting a job once they passed the 
test, since there are few openings in this field. 

Key Principles 

Underlying the practice of assessment in Danish vocational education, and especially the 
assessment characteristic of the practical or "sandwich" phase, are a number of key principles 
that are worth highlighting here. 

First, the approach to assessment assumes the presence of persons who can judge what quality 
is, that is, who have internalized the standards. Because this approach to assessment relies 
heavily on the judgment of the teacher in the school and the mentor in the workplace, it 
requires a deep understanding of the standards by both so that they can recognize whether or 
not a student is performing up to standard. As one team member noted, "The key in the 
Danes' approach to assessment to ensuring that their judgments are reliable is not tests and 
statistics, but the grounding of those responsible for teaching and assessment in what the 
expectations are for students." 

The teacher in the Danish technical school is well informed about what is expected of sUidents 
because he or she is included as a member of the community of practice in which these 
standards are set. The teacher (at least the "practical" teacher) is a skilled craftsman, having 
come through the apprenticeship system. The teacher is also a member of the local trade 
committee, and teachers are represented on the national trade committees. Most teachers 
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spend an average of a week each year in industry in an effort to keep their skills current, and 
they communicate regularly with colleagues in industry. 



Second, assessment is ideally a continuous process embedded in the teaching and learning 
process. The most important assessment takes place daily through an on-going process of 
monitoring of student growth. It depends on continual communication, at one level, between 
the teacher and student, with the teacher providing feedback and encouragement to the student, 
and, at another level, between the teacher in school and the mentor in the workplace, to ensure 
that continuity of teaching and judgment is maintained. 

Culminating assessments are treated not as checks on student learning or as gatekeepers, but as 
an affirmation of achievement by the learner. The journeyman's examination in particular is a 
rite of passage, in which those who have been responsible for the apprentice's development all 
along affirm that the student has met the standards. 

There is a distaste among teachers for tests that seek to check whether students have "mastered 
the material," especially since this is viewed as an abrogation of the authority of the teacher as 
professional. As one team member noted, 

"Generally, the teachers we talked to are not real crazy about testing. Any 
time there is an external test, they see it as a distraction that they just as soon 
would rather not have, because of this general consensus among 
teachers/instructors that 'I know a good product when I see it, trust me to test 
it and to see if it is there or not rather than having it mandated from the 
Ministry of Education or anyone else/" 

We heard a similar sentiment expressed at COTAS, a small electronics firm we visited that 
had succeeded in undertaking the organizational transformation necessary to get ISO 9000 
certification, with dramatic results in terms of increased productivity, improved responsiveness, 
and general alleviation of stress. According to COTAS 's managing director, under ISO 9000, 
the company no longer double checks work; this is a major departure from past practice, 
where inspectors were assigned to check workers work. 

"Now every unit, every worker is responsible for the quality of their own output. It is 
amazing how much people start to improve quality when there is not someone looking 
over their shoulder always." 

Third, assessment should be used not to sort or shut out students, but to show growth and 
guide and encourage them in further learning. Apparently this principle pertains to other 
levels of Danish education, especially in the compulsory school, or Folkeskole, where sorting 
of students on any basis is avoided. According to a teacher at Arhus Technical College, 

"One principle of the Folkeskole teacher is not to focus on the brightest 
students — that would be considered morally wrong; rather the teacher's job is 
to bring all students in the class to perform at a satisfactory level." 
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To bring their students up to standard, Folkeskole teachers use continuous assessment There 
are no grades or report cards. Instead teachers meet with students monthly one-on-one to 
discuss their progress. Teachers have conferences with parents three times a year. At the end 
of the ninth year, all students take an exam, which consists of a written part, prepared by the 
Ministry of Education and taken by students nationally, and an oral part, prepared by the 
students' own teachers. This is not a high stakes examination, since entrance to the 
gymnasium is based on recommendations from a student's teachers. 

Danish youngsters attend the same school from grade one through grade nine. One or two 
teachers, typically the Danish and history teachers, stay with a class of students for the entire 
nine years. Other teachers usually spend at least two years with a class, so the teachers know 
their students very well. At the end of ninth grade, a student's teachers get together and made 
a recommendation, that the student is either 1) "qualified" to go on for study at the 
gymnasium, 2) "possibly qualified" for the gymnasium, or 3) "not qualified." Even this 
recommendation is not final, since a student whose teachers have recommended as either 
"possible qualified" or "not qualified" can still get into the gymnasium by taking a special 
entrance exam. 

We were told that studies have shown that the teacher recommendations are reliable predictors 
of success in the gymnasium. In the United States, where tests are heavily relied upon for 
sorting students and deciding entrance to selective programs, there is a great deal of concern 
that such tests be fair. As a result, American education has devoted enormous energy to 
developing tests that can be scored reliably and cost effectively - so we can get the most tests 
for our money. When we asked Danish educators how they ensure reliability of the 
assessments they make, we were met with blank stares. 

Reflecting on what we had seen in Denmark, we realized that the fairness and reliability of 
assessments are non-issues for the Danes because they don't use tests in making important 
decisions about students, but rather rely on the judgments of teachers. How can they be sure 
that these judgments are reliable, valid and cost effective? The Danes consider them to be 
reliable because they are based not on a single judgment, but on the judgments of many 
teachers and mentors who have known the student for a long time and who have assessed and 
guided the student's progress in an on-going way. The judgments are valid because they are 
made by persons who understand the standards. And they are cost effective because the 
assessment is not some intrusive test designed by outsiders, but is built into the process of 
teaching and learning. 

British Approach 

The systems of vocational qualification that were initiated in Great Britain during the 1980s all 
follow the same basic model of assessment. In this model, evaluation of individual 
performance relies primarily on assessments done internally by teachers or trainers in the place 
where instruction is given. There has been a corresponding reduction in reliance on 
extemally-set examinations, which were commonly used in earlier vocational qualification 
schemes, and are still the dominant form of assessment for academic qualifications. The 
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quality and consistency of internal assessment practice is monitored and improved through a 
process of "verification" or audit managed by the bodies that award the qualifications. This 
section summarizes our findings with respect to methods for assessing individual performance. 
The mechanisms for ensuring the quality of assessment practice are the focus of the next 
section. 

The Assessment Model in More Detail 

The purpose of assessment in the systems of vocational qualification now under development 
in Britain is to provide evidence that the learner has met a standard. All standards are 
specified in terms of competence, or the ability to do something or produce some outcome, 
and include "performance criteria" that indicate the level or quality of competence that need to 
be achieved to meet the standards. The general approach to assessment used under these 
schemes, then, is criterion-referenced as opposed to norm-referenced. 

The heavy reliance on locally-developed assessment by these systems means that a great deal 
of latitude is given to the assessor - whether it be the teacher in the college or the trainer or 
supervisor in the workplace - to decide what evidence to use in assessing a candidate's 
competence and to choose the methods by which this evidence is generated and documented. 

The range of methods that can be used for assessment under the NVQ scheme is shown in 
Exhibit 19, which illustrates the NVQ assessment model. Since NVQs are intended to signify 
competence in work skills, assessment for NVQs typically includes observation of a 
candidate's performance in the workplace. Other approaches are acceptable as well, including 
oral questioning, written tests, and even documentation of evidence of prior learning. 

In the GNVQ system students are required to take a final, externally-developed exams upon 
completing a series of curriculum units, although formative assessment is left to instructors. In 
contrast, SCOTVEC has eschewed the use of final exams with the National Certificate (and 
GSVQs), preferring instead an "additional assessment," which is essentially a project or 
assignment requiring the learner to make use of learning across their entire program of 
learning. Still, it is up to the assessor to design the assessment. Exhibit 20 shows the many 
methods available. 

With the exception of GNVQ final exams, which are paper and pencil tests, all three schemes 
- NVQs, GNVQs and the National Certificate - have a clear preference for performance 
assessment. Assessments that measures one's ability to put knowledge into action lend 
themselves well to systems of qualification such as these that are based on standards of 
competence, that is, the ability to do something or produce some outcome. Performance 
assessments are relatively easy to c t up in the workplace, where the tasks and projects of 
everyday work can be used as situations through which a person can demonstrate competence. 
These sorts of situations do not occur naturally in the classroom; they need to be created. Yet 
most teachers are not trained or accustomed to engage students in purposeful projects that can 
both develop and demonstrate competence. Assessing in this way requires a change in 
teaching. 
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In the GNVQ and SCOTVEC National Certificate programs, which are largely delivered in 
colleges or schools, efforts are being made to encourage teachers to move in this way. For 
example, the guidance materials on assessment put out by SCOTVEC for use by teachers at 
further education colleges urges the use of assessments that involve students in the 
performance of practical projects in which some substantive product or outcome is produced. 

In our visits to further education colleges in both England and Scotland, we did see examples 
of long-term projects being used as means of assessing student mastery of outcomes as well as 
of providing vehicles for student learning. At Cardonald College, long-term projects are used 
to assess outcomes specified in not one, but a series of modules. Also, we were told that in 
order to complete the requirements for Higher National Certificate units in engineering-related 
disciplines students must carry out substantive projects in addition to passing exams in order to 
pass a given module. Students in these programs also are required to carry out a capstone 
project, which is judged by a panel from industry, as a culmination of their course of study. 
How widespread the use of performance assessment is, we cannot say for sure. We got the 
impression that, in the further education colleges of Scotland especially, there are at least 
pockets of expertise and that this expertise is spreading, encouraged by the awarding bodies. 

The results of assessments for each student are compiled in a "portfolio of evidence," which 
enables the student to document his or her progress. NVQ candidates are also required to 
maintain a log, much like the log kept by the Danish apprentice, in which the candidate's 
supervisor indicates which competencies the candidate has demonstrated. The portfolio of 
evidence accompanies the log book as documentation of the accomplishments recorded in the 
log. 

The portfolio of evidence will be a central feature of the GNVQ, as well. Most of the 
evidence that will go into a student's portfolio will be generated through assessments 
developed by his or her teacher. However, students will be required to pass externally set tests 
at the end of each of the mandatory units. The government imposed this requirement, 
apparently with the intent of ensuring that students have covered the necessary content. 
According to the people we met at the Employment Department, the tests in their current form 
are designed primarily to elicit "forced answers." BTEC, one of the awarding bodies, had 
offered a precursor to the GNVQ that was based only on internally-developed assessments. 
BTEC opposed the mandated externally-set tests on the grounds that they will monopolize the 
attention of teachers and students and compromise the benefits for learning of more embedded 
forms of assessment. 

Belief in the salutary effects for learning of embedded assessment (and corresponding belief in 
the deleterious effects of externally-set tests) was a key feature of the Scottish Action plan, 
which, as has been mentioned, was the blueprint for the current system of vocational 
qualifications in Scotland. Thus far, the Scots have resisted returning to the use of external 
exams for vocational qualifications. Such tests remain a prominent feature of academic 
qualifications, however, because they are seen as more reliable and they offer relatively 
inexpensive sorting mechanism for use by universities in admissions. In a recent report that is 
likely to influence government policy, the Howie Committee, which was commissioned by the 
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government to examine the structure of post-compulsory level education in Scotland, 
recommended that vocational qualifications be based on a mixed mode of assessment, with 
both externally and locally-designed assessments. According to Sheila Clarke of the Scottish 
Further Education Unit, the Howie Committee favors this approach at least in part on the 
grounds that it would "assist the parity of esteem between academic and vocational 
qualifications." 

The Model in Practice 

To be certified as an NVQ assessor, a person need only be technically competent in the area of 
competence to be assessed. This helps to ensure the validity of assessment, but it causes 
problems for reliability. In fact, recent studies by the Employment Department indicate that 
there are "some indications of problems with reliability" of NVQ assessments. This comes as 
no great surprise in light of the fact that in the NVQ system, the assessor tends to be the 
candidate's supervisor at work. Such individuals are not likely to have a great deal of 
familiarity with the standards, much less with methods of assessing performance against the 
standards. Assessors are given a single day of training on topics including: familiarity with 
the units and elements of competence; designing assessment situations; and providing helpful 
feedback to the assessee. We were assured at the Employment Department that the recently 
released U.K. -wide standards for assessors would help to lessen the problems of reliability, 
especially as training of assessors is adapted to reflect the new standards. 

From the standpoint of the team, a more fundamental problem for reliability of assessment 
under the NVQs is the reliance on a single person to make the assessment. It is hard to see 
how reliability can be ensured, no matter how well specified the standards or the procedures 
for assessment, if the assessment rests primarily on the judgment of a single assessor without 
the moderating influence that comes from having multiple assessors who are able in some way 
to come to a consensus about the final judgment of whether a standard has been met. In the 
Danish model, for example, the apprentice's performance is assessed through many 
observations by the mentor at work and the teachers at school, who communicate their 
individual judgments on the student's progress either indirectly through the apprentice's log or 
directly, and thus move, over time, toward a reliable judgment ox the apprentice's performance. 
The approach of relying on a single assessor also diminishes the potential of assessment to 
serve as a learning experience, since the candidate only has the benefit of a single perspective 
on their performance. 

Problems of reliability are even more acute when the standards against which performance is 
assessed are broader rather than relatively narrow, occupation-specific skills that characterize 
the NVQ standards. This is the case with the GNVQs and Scottish national certificates. 
Achieving reliability in assessments was the source of much difficulty for the 100 or so 
schools and colleges involved in last year's pilots of the GNVQs. In particular, most sites had 
difficulty getting agreement about where a particular student's work fell on the levels of 
performance specified in the performance criteria. According to an Employment Department 
staff person who oversaw the pilot work, "You've got to know the descriptors and 
performance criteria very well to get it right," suggesting that a lot more needs to be done to 
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increase teachers' understanding of the standards. There was a strong realization that this 
process is going to take a long time. "We haven't gone far down this road," said an 
Employment Department official. 

The Scots have had the most experience with locally-developed assessment, having made it the 
basis for vocational qualifications in the decade since the Scottish Action Plan. According to 
Sheila Clarke of the Scottish Further Education Unit, the benefits that have been reaped 
through this approach include: 

• Teachers and trainers have a clearer understanding of what is to be learned; 

• Use of formative assessment as an integral part of teaching has been strengthened; 

• Assessment is more integrated with teaching and learning, and some of the unfair 
pressures of external testing have been removed. 

Still, the Scots are very much in the learning phase with this approach to assessment A 1991 
review of the Scottish vocational qualification system by the Scottish Office Education 
Department reported that: 

"The greatest single negative influence on learning and teaching was caused by 
the manner in which staff interpreted assessment requirements. In some 
subjects the dominant approach was to teach each learning outcome discretely. 
This led to a fragmented learning experience, and sometimes trivialized it. 
There is a need in some subjects to devise assessment approaches which are 
more sympathetic to the learning process; such approaches would either be 
integrated with the learning activity, or be more holistic, assessing a number of 
outcomes through one instrument. SCOTVEC has encouraged innovative 
approaches to assessment and should continue to do so. Further 
encouragement should be provided through the curriculum and staff 
development activities of colleges and regional and national agencies." (Cited 
in Clarice 1992) 

Teachers are having trouble adapting to new approaches to assessment because it requires not 
just that they learn new methods of testing, but that they teach in an entirely different way. 
According to Sheila Clarke, teacher training and qualifications have not kept pace with 
changing practices of assessment and teaching. 

SCOTVEC has produced some excellent training materials on assessment (as have the 
awarding bodies based in England). Still, these materials and the standards for assessors 
emphasize procedures. While guidance of this sort is necessary, some of those we met in 
Scotland suggested that there is an increasing awareness of the need for assessors to discuss 
actual examples of students' work to ensure consistency of judgment and build understanding 
of the standards. As will be discussed in the next section, the further education colleges are in 



Center for Learning and Competitiveness 

The Role of Standards, Assessment and Credent ialing 



Page 46 



the process of establishing procedures for internal verification of assessment, which in some 
cases at least include internal review and discussion of assessed work by teachers at the 
college. 

Lessons for the U.S. 

► Resist an over-reliance on "end of course" and other on-demand testing. 

"In the school where I was in the United States, there were many more tests, 
too many tests. You have to study very, very hard. There was much stress. 
But after you take the test you can forget about what you learned." 

-- Student, Arhus Technical College 

This was the response of a Danish student in his first year of vocational training who had 
spent a year as an exchange student at a high school in Richmond, Virginia when we asked 
him: "What is the biggest difference between school in the U.S. and in Denmark?" 

When we mentioned this to one of his instructors at the technical college, the teacher smiled 
and said: 

"Yes, we have had several instructors who have been trained in the U.S. And 
we get the same tiling from them: test, test, test, all the time. We say to them, 
'No, no, no, this is not what we do here. If we are testing all the time, we are 
taking responsibility away from the student. Our job is to give responsibility 
for learning to the student.' This is very important for a young person who 
will enter my field [electronics], because an electronics technician is always 
finding new problems to solve in his work, always doing new things. If the 
student does not know how to learn, then they are of no use when they go on 
the job." 

This principle of giving responsibility to the learner as a means of motivating learning was 
also reflected in the practices of the high performance companies we visited. The rejection of 
"double checking" by COTAS, the small Danish electronics firm that had earned ISO 9000 
certification, was mentioned above. In relating the reasons why the company decided to go for 
ISO 9000 certification, Ole Riis Hansen, Managing Director at the firm, told us that: 

"Our company's sales have ^rown considerably since it was founded 15 years 
ago. In the mid-1980s, the company expanded from 22 to 45 employees. I 
and my two partners assumed that because it was a small company, we had a 
handle on what was going on. But we discovered that things had gotten away 
from us. Everyone was constantly rusliing around to keep up with the fast 
growth in the business and the need to always develop new products and 
innovations. We were all very stressed." 
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The results of the changes COTAS has made - including no longer checking over the 
workers* shoulders and instead giving responsibility for quality to everyone — have been 
"amazing," according to Hansen, 



"We are much better able to keep up with the demands of fast-paced work. 
Everyone knows what to do, and there is no more stress." 

At Radiometer, another Danish electronics firm that is a world leader in the production of 
blood testing devices, we noticed the presence of optical scanning devices on every assembly 
worker's station, an indication of the firm's policy to give responsibility to the worker. In 
ticking off some of the most promising human resource development practices used by his 
company, Ian Muir, a forward-thinking training manger at GEC Marconi in Scotland, told us 
that one of the most important was the increasing reliance on self-assessment by workers and 
assessment by peers. At United Distillers in Scotland, which has adopted a total quality 
approach to managing its operations, workers and foremen have more shared responsibility, 
making decisions that affect the day-to-day operation of the plant. 

It only nwkes sense that an education system that aims to prepare young people for 
employment in high performance (and therefore high wage) companies such as these should 
apply to its modus operandi the fundamental principles that underlie practice in such firms. 

► Advocate assessment that is embedded in the teaching and learning process 

On-demand tests intrude on the learning process and discourage students from taking 
responsibility for learning. On-demand tests that are externally set are perhaps even more 
disnjptive because they reflect a distrust in the teachers' judgments. Assessment that is an on- 
going part of the learning process encourages students to take responsibility for learning and 
gives more authority to make judgments to those who are best positioned to evaluate a 
student's progress: his or her teachers and mentors. 

There was a general consensus among members of the team that a student log, like the ones 
used to monitor the progress of Danish apprentices in the school and the workplace, is a 
powerful tool for continuous assessment. As is the practice in the British NVQ and GNVQ 
systems, such a log could be a key part of a "portfolio of evidence" that every student would 
keep to document what they have achieved and what more they need to do toward meeting the 
standards. All of us liked the portfolio of evidence concept, in part because it places 
responsibility for evaluating learning and growth on the learner, with the teacher or mentor 
serving as coach. 

We think that there is much of practical value to be learned from the experimentation with 
continuous or embedded testing that has gone on in Britain, with Scotland taking the lead, 
over the past decade. The British have produced a scries of excellent guidance pieces on 
alternative assessment that could be used as models for American educators seeking new 
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approaches to assessment One example is "The National Certificate: A Guide to Assessment," 
put out by SCOTVEC. In general, we suggest that American educators keep an eye out for 
any new developments in the area of assessment coming from the British Isles. 

► Encourage schools to involve members of the community in assessing students 

The journeyman's examination may be in effect a rite of passage, but the fact that 
representatives of the partners come to the school to judge the apprentice's work carries 
important messages for the student, the teacher and the school. 

Schools and colleges in the U.S. should take advantage of the expertise and interest of 
members of the community by involving community members in assessing student work. This 
would make especially good sense when the work to be judged involves a long-term project - 
for example, a capstone piece of work by older students - or portfolios of students' 
accomplishments over time. 

This practice would have a number of benefits. The presence of outside judges would 
encourage students to do well and give teachers an incentive to ensure that their students 
excel. If well run, the experience could be more than just an assessment, but could provide 
opportunities for learning by student and teacher alike. As in the examination of the Danish 
apprentice, outside judges could help students gain insight into their work by asking questions 
that encourage students to reflect on how they did what they did, where they made mistakes, 
and what they learned from their mistakes. Teachers would benefit by seeing the standards 
against which outsiders judge their students. This practice would provide a more direct and 
strategic way for employers and other community members to contribute to student learning 
than is the case in most school-community partnership programs today. 

► Provide training and support to help teachers learn new methods of assessment; but 
don't expect quick results. 

In 1983, the Scottish Action Plan set forth a bold vision of a system of assessment for 
vocational education based not on external exams but on assessments developed locally by the 
teacher or trainer who provided the instruction, assessments that ideally would be embedded in 
the teaching and learning process. Ten years later, much has been accomplished, but there is 
still a long way to go before there is a level of comfort with this approach throughout the 
vocational education system. While there has undoubtedly been resistance among teachers and 
trainers to make changes, the main reason that it has been so difficult to spread is surely the 
magnitude of the changes it requires in practice. For this new approach to assessment to be 
effective in benefiting learning, it needs to be accompanied by thorough changes in teaching. 

Heeding this lesson from the vanguard work in Scotland, the Scots and the British generally 
are taking steps to help teachers make the changes in practice necessary to accommodate new 
approaches to assessment In particular, they have produced a wide range of training and 
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guidance materials that seek to clarify the standards and delineate procedures for assessment. 
Any effort that would hope to succeed in the United States would have to at least match these 
materials in quality. 

Still, the British are coming to realize that training in assessment procedures is not enough; 
that to come to a true understanding of the expectations set for students by standards and to 
make reliable judgments of student performance against such standards, teachers need to be 
able to examine and discuss with other teachers actual samples of student work. This 
discussion of student work needs to be incorporated as an integral part not just of teacher 
training, but of the everyday practice of teaching. The British are experimenting with ways of 
doing this through procedures for internal verification of assessment (which are discussed in 
the section that below). 

► Give teachers opportunities for professional development that help them gain a clearer 
understanding of what students need to be able to do to succeed. 

Having clearly stated standards of what students should be able to do to prepare for productive 
roles at work is a critical first step to enabling teachers to make reliable assessments of their 
students. Teachers in the vocational education systems of Denmark and Britain have clear 
standards to guide them in making assessments. Teachers in this country do not. They have 
to rely on indirect signals from textbooks, standardized tests, and packaged curricula. In 
setting such standards in the U.S., it will be important to involve teachers as active participants 
in the process. 

However, for teachers to understand what is expected of students in the workplace outside of 
school, they need to have on-going exposure and contact with that world. One way to help 
teachers make the link is to involve professionals from outside the schools in related 
disciplines and fields in assessing their students' work. Teachers would also benefit from 
working directly in related fields through internships and sabbaticals. The "practical teachers" 
in the Danish vocational education system spend an average of a week each year working in 
industry in their fields. The British are investing considerable sums to create similar 
opportunities for their teachers, including academic teachers. 

Still another way to improve the understanding among educators of the expectations of the 
"outside world" for students is to encourage people to enter teaching who have experience 
outside of education. 

2. How is the quality of vocational education programs monitored and improved? 

Main issues of interest to the team: 

• How to ensure accountability of vocational schools and programs in systems where 
responsibility for assessment is primarily local; 
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• How to structure accountability systems to promote continuous improvement in the 
system. 

Danish Approach 

The quality of Danish vocational education is controlled through the same structure through 
which standards for the system are defined and updated. The mechanism for monitoring 
system quality is continual communication among parties involved in the structure at all levels 
with a focus on improving the capacity of the system to prepare skilled woricers. 

An important vehicle for this kind of communication at the local level is the apprentice, who 
in going back and forth between school and workshop, serves as a kind of feedback 
mechanism. When this "intermediate product" is not satisfactory coming from one side or the 
other, the other is alerted and steps are taken to remedy the problem. 

Another interesting instance of the use of the apprentice as an "instrument" for monitoring 
system quality is the practice - evidently fairly common - of sending apprentices for training 
stints in other countries. Beyond the advantages to the student, this practice allows the Danes 
to determine whether their own system is up to the standards of the training offered in other 
countries - in essence, to engage in international benchmarking of standards and practice. 
Evidently this practice also encourages technology transfer. An employer association 
representative told us about a group of Danish construction apprentices who, after a training 
stint in the U.K., brought back with them methods that they convinced their Danish employers 
were superior to the ones they had been using. 

Danish companies involved in vocational education seem to keep very close tabs on the quality 
of the technical college programs in which their apprentices are enrolled. Some companies 
conduct periodic reviews of these programs. For example, a foreman at Radiometer told us 
that after a recent visit to a machining program at a local technical college, he and his 
colleague recommended to the college that it broaden the focus of the training in the program 
from the machining of discrete parts to production as a comprehensive and continuous process. 
According to a training manager at Danish Railways Workshop, larger companies send 
apprentices to different colleges in an effort to "shop around" for the best programs. This has 
the beneficial effect of encouraging competition among the colleges. 

And, of course, skilled workers frequently go to the colleges to judge journeyman 
examinations as representatives of the partners, a practice encouraged by their companies. 
Some skilled workers travel all around the country judging these exams, which gives them a 
good basis for comparing the quality of programs. Skilled workers as well as instructors from 
the technical colleges, also make up the membership of the local trade committees, which 
advise on technical college programs in their field. The local trade committees are represented 
on the Council for Vocational Education, which facilitates the flow of information about 
problems and innovations on the local level to the national level where the goals and broad 
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frameworks for the system are set. Communication from the top down builds a common 
understanding of the standards and ensures that the national standards are consistently applied 
throughout the system. 

In addition, the partners have a direct handle for holding the technical colleges accountable 
through their representation on the governing boards of the colleges. The 1991 reforms sought 
to increase competition among the technical colleges. The colleges are now considered to be 
self-governing institutions, but the bulk of their funding comes from the national government 
based on the number of students they enroll. Hence, the colleges have an even greater 
incentive to satisfy the needs of their customers: the partners and the local community. 

British Approach 

In Britain, there is not, as there is in Denmark, a history of coop** ~tion among business, labor 
and education that supports on-going communication among the various parties at various 
levels aimed at monitoring and improving the system's capacity to turn out well-trained 
workers. Lacking such an infrastructure of established relationships, the British are having to 
create by bureaucratic means a system for ensuring the quality of vocational education. 

The key challenge for developing systems of vocational qualification in Great Britain is how to 
create mechanisms for ensuring that national standards for qualifications are being consistently 
applied in systems where control over methods of teaching and assessment is primarily at the 
local level. The British evidently believe that the standards and procedures for quality control 
themselves need to be applied on a consistent basis nation-wide, since they have secured 
agreement by the various awarding bodies to follow the same, comprehensive approach to 
quality assurance. The main features of this approach are: 

Validation of Awards - Ensures that standards are clearly reflected in each awanl. 

Training and Certification of Assessors - Ensures that assessors are qualified to carry 
out assessments according to recently-released national standards for assessors and 
verifiers. 

Approval of Centers - Checks the capability of new "centers" where instruction and 
assessment would be provided to support candidates from enrollment to qualification. 

Verification - Ensures that procedures for assessment are being consistently applied 
and assessors are working toward the same national standards across centers. 

It was evident to our team that these mechanisms could well be used by the national awarding 
bodies as instruments for enforcing quality in a top-down manner. However, there seems to be 
widespread consensus in Britain that this is not the way to go, in part because it would reduce 
the flexibility of the system, but also because the resources are not available to support a 
regime of top-down control. The best a national system of quality assurance can do is to take 
steps to ensure that, at the local level where teaching and learning take place, the standards are 
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well understood and the tools of quality measurement - in particular methods of assessment - 
are being applied consistently and effectively. So that national involvement won't be 
perceived by the locals as threatening their autonomy, this has to be done with an attitude of 
support. 

An innovative approach to accomplishing this is the system of verification now under 
development in Britain. This will be the focus of our analysis in the rest of this section, since 
we think that it has features that could be incorporated into a system for use in this country. 

The verification process focuses on the assessment process in and across the "centers" where 
training and assessment for qualifications is provided. It has two dimensions: 

Internal Verification - ensures that all assessors within a center are making consistent 
judgment by following accepted assessment procedures. 

External Verification - ensures that assessors within a center are making judgments 
consistent with national standards. 

Internal verification is carried out by a designated person within the center who serves as a 
combination of internal coach, critic and cheerleader. This internal verifier supervises the 
assessments carried out by his or her peers, reviewing samples of assessed work and providing 
training and support. The further education colleges, especially those in Scotland, seem to be 
ahead of centers located in business or community sites in providing support and quality 
assurance through internal verification. At least some further education colleges in Scotland 
have set up formal procedures for verification of assessment, including the standardization of 
assessment processes and internal reviews of assessed work. In certain cases, groups of 
instructors come together to re-evaluate student work that has already been assessed. This 
process of re-examining, discussing and then seeking to reach consensus on actual pieces of 
student work helps to build understanding among the teachers about what it takes to meet the 
standard. 

In the external verification process, persons certified by an awarding body periodically visit 
e;> h center where they conduct a one-day review of assessment procedures. Following a 
standard review process, the external verifier asks to see samples of assessments with student 
work, observes assessments in progress, and examines the operation of the internal verification 
system. Exhibit 21 is a report form used by external verifiers of qualifications awarded by 
SCOTVEC to put a "hold" on the certification by SCOTVEC of a particular award module. 
The "reasons" listed on the report suggest the kinds of criteria the verifier uses in making a 
review. Of particular concern to the verifier is the kind and quality of evidence used to decide 
that a student has met a given standard. Verifiers typically require that the samples of 
assessment and student work submitted for review include justifications of why the evidence 
presented warrants a determination that the standard has been met. 

The centers tend to look upon external verification as an audit, a check by the awarding body 
to make sure the center is not doing anything in violation of standard procedures. According 
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to those at SCOTVEC, however, the awarding bodies see the purpose more as providing 
constructive feedback. From their perspective, the External Verifier is supposed to be not just 
a critic but a source of advice and support. We saw some evidence that this is the case An 
Assistant Principal at Monkwearmouth College in Sunderland, England, told us tnat external 
venfication is a painful process for the college - particularly since it is necessary to persuade 
the external venfler of the value of the evidence used to judge actual student work - but that 
in the final analysis, it is a useful one. At Vaux Breweries in Sunderland we were told by 
human resource staff that the external verifiers who have been sent to review Vaux's NVQ 
programs have all been people with experience in their industry, and that the feedback they 
have received has been quite useful. 

At SCOTVEC, some concern was expressed over the difficulty of recruiting enough people 
from industry to serve as external verifiers for school-based programs. Of SCOTVEC's 470 
external verifiers, 65 percent are educators and 35 percent are from industry. We did see 
evidence of the potential for a burgeoning cottage industry of freelance consultants who serve 
as venfiers. This may not be all bad, but it does not help to increase involvement of industry 
in assunng the quality of school or college programs intended to prepare for work in industry. 

Of even greater concern is whether the system can afford to field enough verifiers to cover the 
ESS^T*? 1, ° f ( l ualifications and centos where vocational qualifications are offered. 
SCOTVEC esnmates that it would need over 1,000 verifiers to have enough to do the woik for 
every cognate group" of modules - this for a country with a population and land mass the 
size of Oregon. Not all of these would need to be full-time - in fact, most verifiers work on a 
part-time basis, averaging around 15 days a year. Still, the resources required to finance this 
system are considerable, especially when one figures in the cost of training and supporting all 
ot these people and ensuring that they are doing a quality job. 

In fairness to the British, it was pointed out by members of our team who have had experience 
with systems of accreditation for professional and technical occupations and colleges and 
universities in the U.S., that these systems employ legions of external reviewers. Still, the 
business people we met with in Britain in general seemed uneasy about the level of 
bureaucracy involved in the British vocational qualification systems, both in terms of the high 
cost of financing "an army of verifiers," and the potential that the requirements of verification 
could be a drag on their efficiency. 

Our sense is that, on balance, the British are taking the right tack to avoid over-bureaucratizing 
their approach to verification. They are doing this in at least two ways. First, they are 
keeping the focus of the external verification very narrow, confining it to the process of 
assessment, and in particular, the evidence that is used to determine whether or not a learner 
has met a given standard. The external verifiers do not spend their time calculating the 
student-teacher ratio or counting the number of books in the library, but instead focus on 
student woik produced to meet specific standards. In other words, the verifiers check to see 
whether the program is truly producing the outcome it claims. This gives a great deal of 
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Lessons for the U.S. 

Don't let the drive for accountability overwhelm learning. 

Z1£%Z Attests TIT T ***** in COnvincin * 

-- New Standards Team Member 
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In the U.S., external tests are still seen as the most effective way of ensuring that students and 
schools are performing up to standard. If anything, pressure for external testing has increased 
in recent years from politicians and members of the public eager to "hold the schools 
accountable." 

If the goal of education is to enable students to take responsibility for learning, the system 
mus-. practice what it preaches. A key lesson from our study of vocational education in 
Denmark and Great Britain is that we cannot hope to encourage students to take responsibility 
for teaming if we are constantly giving them tests to see fiow much they have absorbed. And 
we can't expect teachers and other educators to take initiative to find better ways to improve 
student learning if we are constantly looking over their shoulders. 

► Consider an approach to quality assurance based on the best of the British verification 
system model 

The scheme of internal and external verification that is being developed as the main quality 
assurance mechanism in the emerging British vocational qualification systems has much to 
offer a model of vocational education in the U.S. The model is not that of the certification by 
professional associations characteristic of fields such as health care in the U.S., or that of the 
institutional accrediting bodies familiar to U.S. higher education. Rather, to avoid the 
patronage and parochial interests that characterize such bodies, a system of quality assurance 
for workforce education would have to be coordinated by a single national body that 
transcends industry or professional affiliations. In Great Britain, such a role is played by 
SCOTVEC for the Scottish vocational qualifications, and NCVQ for the U.K.-wide NVQ and 
GNVQ systems. Without such an umbrella organization in the U.S., it will be impossible to 
set national standards, let alone ensure that national standards are being met. 

In the U.S., the quality of training programs is guaranteed through accrediting bodies. 
Currently, to receive and maintain accreditation, educational programs must meet accreditation 
standards, which, typically, are not based on outcome measures but instead on institutional 
characteristics that serve as proxies for quality, such as, credentials of the faculty, number of 
books in the library, duration of the course, course structure, content, etc. All of these bodies 
employ part-time accreditors to ensure that institutions meet these criteria, which bear a weak 
relationship to quality student outcomes. 

In the model we envision, external verification would be focused on outcomes, and in 
particular on the evidence by which outcomes are assessed. This approach of verification 
would allow for much more flexibility over where and how learning takes place. Flexibility is 
constrained by accreditation systems common to the U.S., which focus on inputs and contents. 
One member of our team warned that, "The minute we in the U.S. start to say, 'make sure that 
they are also teaching such and such, and make sure that they do things this way,' that is when 
we start to get into trouble." 
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CONCLUSION 



There is no question of the need for a coherent system for school-to-work transition in the 
United States. Many observers have pointed out that the U.S. is the only industrialized 
country without such a system. Lacking clear guidance on what it takes to succeed in the 
world of work, many if not most young people entering the workforce in the U.S. flounder 
throughout their late teens and early twenties, unable to find employment offering a decent 
wage and opportunities for advancement What education and training they do pick up along 
the way does not prepare them for jobs in the most high performance workplaces, 
organizations that have transformed themselves to respond to the challenge of rapid economic 
and technological change and that require broadly skilled workers to run them. 

The Danes and the British are struggling to adapt or create systems of education and training 
that will prepare workers capable of thriving in such high performance workplaces. We 
believe that certain features of the approaches to standard setting, assessment and quality 
assurance being tried out by these countries should be part of the "vision" for a cohesive 
school-to-work transition system in the U.S. These features include: 

National Standards for Workplace Skills 

• Set by industry coalitions that include all relevant stakeholders; 

• Defined in terms of competence what a person should be able to do - based on 
an analysis of the competencies that are be needed by workers in high performance 
workplaces; 

• Couched in terms of broad frameworks with standards for "core skills" integrated 
with standards for technical competence. 

Employer "Buy-in" of Standards Development and Training 

• Cultivated at both the national and local level to build the "market" for standards 
and to make standards-based credentials widely recognized, valued and sought 
after; 

• Targeted at ensuring that standards reflect the skills employers need and building 
on-going employer demand for workers trained to standards. 

National Board to Oversee the Setting and Quality Control of Standards 

• Reviews and approves all standards for technical education and training; 

• Manages the external quality assurance of programs by auditing internal 
assessment processes; 
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Local Assessment of Student Performance 

• Based on judgments of teachers, mentors and other adults, including members of 
the community, as well as peer and self-evaluations; 

• Documented in a portfolio of evidence, structured according to the national 
standards, that is assembled by each student with guidance from teachers and 
mentors; 

• Audited locally by panels of teachers who re-score samples of previously assessed 
student work. 

External Quality Assurance 

• Based on an audit of assessment procedures and evidence used to demonstrate 
competence from a representative sample of student work; 

• Focused on ensuring fidelity of evidence to national standards; 

• Aimed at providing guidance on, not control over, assessment procedures. 
Training and Support for Teachers 

• Empowerment of teachers to function within broad guidelines as self- directed 
teams; 

• Training in assessment using actual samples of student work that illustrate 
benchmark standards of performance; 

• Support from administrators for schedule changes, release time and the like to 
accommodate new approaches to teaching and assessment; 

• Training in project planning and management and other skills as needed to help 
teachers succeed with new teaching and assessment methods; 

• Internships in non-school workplaces and other professional development activities 
aimed at helping teachers get a clear idea of what their students need to be able to 
do to succeed in work and life beyond school. 

If the U.S. is to be successful in educating a productive and prosperous workforce, the vision 
for reform must encompass more than the creation of a viable school-to-work transition 
system. 
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For one, we need to ensure that our basic education system provides the necessary foundation 
for programs designed to prepare young people for work. The programs we observed in 
technical colleges in Denmark and Britain seemed to us to be of similar caliber as some of the 
high quality technical training programs offered by our best community colleges. The only 
difference is that the majority of students in their technical college programs are 17 or 18 
years old, whereas the average age of students in advanced technical training in U.S. 
community colleges is in the late 20s. Especially in Denmark, the need for remediation at the 
technical college level is low. Danish students leave the compulsory school at age 16 fluent in 
English, well-grounded in both applied and theoretical math, and having completed four years 
of laboratory physics. 

Thanks to the superior preparation they receive in compulsory school, the majority of students 
in Denmark and many students in Britain are able to enter into high level technical training at 
an age when most of our students are still in high school. Unless we in the United States can 
eliminate this deficit -- that is, unless we can bring our students to a similar level of readiness 
by around age 16 or 17 -- our system of technical education and training will forever be 
"remedial" in comparison with that of other countries. This same realization led the 
Commission on the Skills of the American Workforce, in its 1990 report, America's Choice: 
High Skills or Low Wages!, to propose a Certificate of Initial Mastery (CIM). The CIM 
would be benchmarked to international standards of performance for 16-year-olds, though it 
could be earned at any age. A young person receiving a CIM would be assured that he or she 
is prepared to enter world-class technical and professional training or a world-class university 
preparation system. 

Neither the Danish nor the British are satisfied with their basic education systems - despite 
the advantages they hold over ours - and major reforms are underway in both countries. 
While basic education was not the focus of our investigation, we did leam enough about the 
reforms underway in these countries to recommend that they be studied by American educators 
involved in the restructuring of basic education in the U.S. 

Realizing the vision of a system of education in the U.S. that prepares young people to be 
high performance workers will require not only changes at the level of basic education, but a 
rethinking of our university preparation system as well. Both countries are making changes to 
their university preparation systems aimed at ensuring that young people educated in this way 
develop skills that meet the requirements of industry as well as academia. The move in 
academic education in these countries is also toward integrating theoretical and applied 
learning in teaching and building bridges with vocational programs to allow greater flexibility 
of choice by students. 

In Denmark and Great Britain, we saw the promise of a standards-based system of vocational 
education and training that is successful in preparing for work in a global economy. Where 
the systems of these countries work best, a critical factor is the involvement of employers, 
whether through the training of apprentices, contributing to the development of standards or 
curricula, or using standards to improve employee satisfaction and performance. Employer 
involvement is no panacea. Employers do not speak with one voice and at times give 
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conflicting messages, for example, by advocating that education should focus on 
"employability and adaptability," but then hiring the classically trained, narrowly-proficient job 
candidate. As the final customer of workforce education and training, however, employers 
must play a central role in developing and sustaining a system of workforce preparation. Still, 
employer leadership should be cultivated in a way that facilitates communication and 
cooperation at every level of the system - national, state and local — between all stakeholders, 
including workers, educators and .policy-makers. Our study of vocational education in 
Denmark and Great Britain has persuaded us that it is only on such a foundation of 
communication and cooperation that we in the U.S. can hope to build a system for preparing 
young people to be high performance workers of the future. 
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APPENDIX 



Study Team Members 

The Standards Team was headed by Mr. Davis Jenkins, coordinator of the New Standards 
Project's Applied Learning initiative, a part of the Learning Research and Development Center 
of the University of Pittsburgh. A graduate of Princeton University, Mr. Jenkins is currently 
pursuing a Ph.D. at Carnegie Mellon University, where he is conducting research for a 
dissertation on human resource practices of U.S.-based Japanese manufacturing "transplants." 
Before pursuing graduate school, he worked for the Academy for Educational Development, an 
international consulting firm. His many assignments at the Academy included assistance to 
community colleges and small manufacturers on the design of "total quality" approaches to 
training for advanced production systems. 

Ms. Susan Goldberger, a senior project manager for Jobs for the Future (JFF), coordinates the 
JFF health industry work-based learning initiatives. She is also responsible for program 
evaluation and is conducting a two-year fomiative evaluation of a health industry youth 
apprenticeship demonstration project. Ms. Goldberger has worked for many years as a health 
policy consultant to consumer and labor organizations in Massachusetts. The recipient of a 
national Pew Health Policy Fellowship, she is completing her doctoral studies at the Heller 
Graduate School at Brandeis University. 

Ms. Samantha Guerry, the Associate Director of Policy and Programs for the Center for 
Learning and Competitiveness (CLC), was a principal manager of the Comparative Learning 
Teams Project. Her research and travel in Scandinavia and Europe investigating systems of 
education, training and economic development laid the ground work for the teams' visits. She 
works with international representatives from business, education, labor and government to 
implement CLC programs on a wide range of issues. Formerly, Ms. Guerry was the Program 
Director for the New American Schools Development Corporation, where she worked with 
America's leading corporations, practitioners and policymakers on dramatic, comprehensive 
reform of the K-12 education system. She has also worked as a communications and strategic 
planning consultant for corporate and national non-profit organizations. 

Ms. Sonia Hernandez is State Director of Education Policy in Texas. As a chief policy aid to 
Governor Ann W. Richards, she helped to formulate a comprehensive legislation package 
known as the "smart jobs" initiative which was recently introduced by the Governor. Before 
joining the Governor's office in 1991, she was Vice President of the National Center on 
Education and the Economy where she directed the Center's initiatives aimed at restructuring 
the Rochester City School District. Concurrently, Ms. Hernandez was an adjunct professor at 
the University of Rochester. She began her career as an elementary school teacher and was 
Teaching Principal of Emma Frey Elementary School in San Antonio's Edgewood Independent 
School District. She is a board member of the National Board for Professional Teaching 
Standards and was named 1984 Hispanic Woman of the Year, Southwestern United States 
Region. 
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Dr. Martin Nahemow, Director of School to Work Programs at the Learning Research and 
Development Center, University of Pittsburgh, has a Ph.D. in Physics from the Polytechnic 
Institute of Brooklyn, and for 23 years was Fellow Srientist at the Westinghouse Corporation 
R&D Center. He is the author of numerous scientific publications and hoids three patents. 
While at Westinghouse, Dr. Nahemow was active in the secondary science education programs 
of the Westinghouse Foundation. At LRDC, he has overseen the development by teams of 
teachers and employers from throughout Pennsylvania of the curriculum and teacher training 
for the Pennsylvania Youth Apprenticeship Program, a demonstration effort funded by the U.S. 
Department of Labor and the Pennsylvania Commerce Department to serve as a model for a 
national youth apprenticeship program. Recently, Dr. Nahemow received funding from the 
Sloan Foundation to produce a handbook and teacher training curriculum on using portfolio 
assessment in youth apprenticeship. 

Mr. Wayne Neuberger, Assistant Superintendent for Assessment and Technology for Oregon's 
Department of Education, has played a key role in implementing the Oregon Educational Act 
for the 21st Century, an ambitious effort to improve the competitiveness of Oregon's 
workforce by raising standards for all students and easing the transition from school to work 
for young people not headed directly for college. In particular, his office has been responsible 
for translating recommendations from numerous task forces and hearings throughout the state 
into a list of outcomes standards for the certificates of initial ana advanced mastery called for 
under the refonn act. His office is now orchestrating the development of an assessment system 
to measure student performance in relation to these outcome standards. He has extensive 
experience in educational assessment, having served as Director of Research and Evaluation 
for the Beaverton School District then as Director of State Assessment for Oregon. 

Mr. C. Gary Standridge , Director of Research and Planning, Fort Worth Independent School 
District, Texas, directed a two-year study of skills needed for success at work that was based 
on analyses of 3,500 jobs in over 300 companies and community organizations in Fort Worth. 
The data from this research showed that young people entering the workforce must be 
prepared to handle much more than the basics of reading, writing and math; they must be able 
to communicate effectively and think critically. These findings led to the creation of Project 
C3, a partnership of community, corporations and classrooms that is seeking to teach students 
real-world skills by connecting instruction in school with applications in the workplace. This 
multi-faceted initiative has been featured on national television and has been cited as an 
exemplary initiative by the U.S. Department of Labor, the National Alliance of Business and 
the American Business Conference, and other national and state organizations. He has received 
frequent invitations to speak to districts and states throughout the country that want to emulate 
the Fort Worth approach in their locales. He was an advisor to the U.S. Secretary of Labor's 
Commission on Assessing Necessary Skills (SCANS) and is currently a member of the 
Technical Design Committee for ACT's SCANS Assessment Project 

Ms. Bess Stephens is the Corporate K-12 Education Relations Manager for programs of 
Hewlett-Packard (HP). A twelve-year HP veteran, she assumed this role in the Government 
Affairs department in September 1989, when the company created the new position. Her 
responsibility is to bring strategic direction and leadership to the company's growing interest 
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and involvement in K-12 education. Since starting HP's K-12 function, she has led the 
development of the company's strategy for its education program, brought focus, leverage and 
impact to its existing activities and established several new initiatives. She also advises HP 
managers on education issues, serves as a consultant to HP people who are participating in 
partnerships and represents HP with educators, legislators, community organizations and trade 
associations. 

Ms. Cheryl Fields Tyler, Director of Workforce Excellence for the American Electronics 
Association (AEA), oversees all of AEA's activities related to workforce quality and "high 
performance" work organization. She is founder and director of AEA's national flagship effort 
to develop voluntary skill standards for high technology industries. With funding from the U.S. 
Department of Labor, she has built a coalition of AEA member firms, educators, high tech 
workers and government officials to carry out this initiative. She staffs three major national 
AEA committees concerned with workforce skills and development and directs AEA's 
Electronics Education Foundation. In addition to her national role, she provides program and 
policy support to AEA education efforts at the local and state level. She has served on 
numerous advisory councils related to education and workforce issues, including the National 
Association of Manufacturers Workforce Policy Committee and has provided expert testimony 
on developing skill standards and other high performance work issues to various policy bodies, 
including the U.S. Senate and House of Representatives. Before joining AEA, Ms. Fields Tyler 
worked for NASA's Jet Propulsion Laboratory, where she was responsible for communications 
and policy strategy for the Galileo and Ulysses space exploration missions. 
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Exhibit 2, page 1 of 2 (Denmark) 



Extracts from the Ministerial Order on the Vocational Education and Training 
Courses in the Field of Mechanical Engineering 



School periods 


2nd 


3rd 


4th 


5th 


6th 


BASIC SUBJECTS - 
number of weeks 




in 


j.j 


L.L 


i 
i 


Trade knowledge and 
technological development 


* 


* 








Workshop techniques, 
60 lessons 


* 


* 


* 


* 




Knowledge of materials 


* 


* 


* 






Natural Sciences 


* 


* 








Foreign language (B), 
English 


* 


* 








Danish, 60 lessons 




* 








First aid/fire fighting 




* 








Mathematics 












Labour market relations 


* 


* 


* 






Production and service 


* 


* 


* 


* 




Communication technology 






* 


* 


* 




School periods 


2nd 


3rd 


4th 


5th 


6th 


AREA SUBJECTS- 
number of weeks 




6.3 


4.7 


1.5 


22 


Thermal joining 


* 


* 








Assembling and repair 
techniques 


* 


* 








Machining with and without 
shavings 




* 


* 


* 




Control techniques 




* 


* 


* 


** 


Measuring techniques 


* 


* 








Data techniques 


* 


* 
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Exhibit 2, page 2 of 2 (Denmark) 



The 2nd School Period 

10. (1) It is the aim of the teaching in the 2nd school period 

1) to provide the student with basic skills in turning, milling, hole making, grinding, measuring 
techniques, thermal and mechanical joining techniques, assembling and repair techniques, 
manual machining with/without shavings as well as control techniques, 

2) to provide the student with knowledge about the theory of the trade which relates to the 
respective work disciplines, 

3) to provide the student with knowledge about and skills in ordinary first aid, 

4) to provide the student with knowledge about the trade and about the technological development, 
including production and service, as well as 

5) to provide the student with knowledge about the social development of his time, including 
labour maricet relations. 

(2) the teaching in the area subjects (part jobs) in the 2nd school period has the following 
objectives and other framework with regard to the contents: 
1) Thermal joining 

Thermal joining of construction parts. The work shall appear without any visible faults. The 
following work disciplines shall be included and the following requirements met: 

a) Arc welding 

The student shall be able to weld from the top down and choose electrodes according 
to welding disciplines and materials. 

b) Inert gas welding 

The student shall be able to weld together pipes and thin plate constructions. 

c) Hard soldering 

The student shall be able to carry out hard soldering in steel and copper mate- Ms. 
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Exhibit 4 (Great Britain - NVQ) 



Example 2: functional analysis from commercial horticulture 

The advantages of functional analysis as a method of developing standards are 
illustrated in the following example from commercial horticulture. 

The example illustrates how: 

• Analysis of functions emphasises the objectives of the activity and so is more likely 
to include broader factors such as task management and contingency management 
Task analysis can easily lead to the development of standards for separate tasks only. 

♦ The systematic, top-down approach can clarify relationships between different 
functions, and can be used to ensure that different units and elements do not overlap. 

The example shows a cohesive structure of units and elements derived by 
functional analysis for one key purpose of commercial horticulture "provide 
ornamental beds and borders". This is contrasted with a list of tasks and skills, written 
for the same area of work but using a bottom-up approach. There is also an 
illustration of how the job competence model might be used to incorporate the broader 
aspects of competence. 

Functional analysis 



Provide Ornamental Beds and Borders 



Units 



r 



Propagate 
softwood. 



1 1 

Produce plants Establish 

from seed ornamental 

semi-ripe. beds and 

and hardwood borders 

cuttings . 



1 

Maintain 

ornamental 
beds and 
border 



Maintain 
safety when 
using hazardous 
chemicals 



I 

Etc 



Elements 

• Prepare uncultivated ground for use as 
bed/border 

' Prepare beds and borders for planting and 
replanting 



Plant seeds and borders to planting plans 
"~ Maintain tools used for propagation 
'-Etc 



"™ Prepare hazardous chemicals, sprays 
and applications 

~" Use common hazardous chemicals 
and applications 

— Dispose of waste . 
hazardous chemicals 

'-Etc 



— Weed a bed/border by hand and/or using tools 
Maintain the appearance and heaJth of plants 

— Maintain the soil condition and physica' 
appearance of bed/border 

'—Etc 
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Exhibit 5 



(Great Britain - NVQ) 
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Exhibit 6 (Great Britain - NVQ) 



The NVQ framework 



lost* ■■ 



nrow 



Level I 




ine** 



Areas of competence 

The NVQ framework 



Each NVQ is given a title and a level to place it in the NVQ 
framework. The NVQ framework shows how qualifications 
relate to each other and how people can progress through the 
system. 

Areas of competence 

The primary purposes of the NVQ framework are to create a 
coherent classification for qualifications and to facilitate transfer 
and progression, both within areas of competence and between 
them. 

The main headings for the NVQ framework classification result 
from a functional analysis of work roles, and provide the initial 
organising structure for competence -based qualifications. Further 
refinements to the system are being made as the qualifications 
are developed and routes for progressionAransfer are identified. 



•veW 1 



Levels 

The following definitions of the NVQ levels provide a general 
guide and are not intended to be prescriptive. 

Level 1: competence in the performance of a range of varied 
work activities, most of which may be routine and predictable. 

Level 2: competence in a significant range of varied work 
activities, performed in a variety of contexts. Some of the 
activities are complex or non-routine, and there is some 
individual responsibility or autonomy. Collaboration with others, 
perhaps through membership of a work group or team, may often 
be a requirement. 

Level 3: competence in a broad range of varied work activities 
performed in a wide variety of contexts and most of which are 
complex and non-routine. There is considerable responsibility 
and autonomy, and control or guidance of others is often 
required. 

Level 4: competence in a broad range of complex, technical or 
professional work activities performed in a wide variety of 
contexts and with a substantial degree of personal responsibility 
and autonomy. Responsibility for the work of others and the 
allocation of resources is often present. 

Level 5: competence which involves the application of a 
significant range of fundamental principles and complex 
techniques across a wide and often unpredictable variety of 
contexts. Very substantial personal autonomy and often 
significant responsibility for the work of others and for the 
allocation of substantial resources feature strongly, as do personal 
accountabilities for analysis and diagnosis, design, planning, 
execution and evaluation. 
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Exhibit 7 (Great Britain - GNVQ) 



TIME TABLE FOR PHASING-IN GNVQs 



1992-3 



Health & Social Care 

Leisure & Tourism 

Business 

Art &r Design 

Manufacturing 

Science 

Construction 8c Built Env 
Catering & Hospitality 
Information Technology 
Engineering/Technology 
Media/ Comms/Perf Arts 
Agriculture/Environment 
Distribution 
Management 



199S-4 



1994-5 



1995-6 




9 
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Exhibit 8, page 1 of 2 (Great Britain - GNVQ) 



MANUFACTURING GNVQ - LEVEL 2 - SUMMARY OF UNITS 
AND ELEMENTS 

2ML PRODUCE AND APPLY STANDARD COMMUNICATION METHODS 

1.1 Design a product from a given design brief 

1.2 Apply standard communication methods 



2M2. PRODUCTION PLAN 

2. 1 Investigate manufacturing systems 

2.2 Produce production plans for products 

2.3 Prepare production schedules 

2M3. PROCESS OPERATIONS 

3.1 Prepare materials/components and equipment/machinery 

3.2 Operate material processing 

3.3 Assemble materials/components to specification 

3.4 Inspect manufactured products 

2M4. QUALITY, SAFETY AND THE ENVIRONMENT 

4.1 Apply quality assurance and control systems 

4.2 Identify the health and safety requirements for the individual and the workplace 

4.3 Identify the environmental effects of production processes 



Center for Learning and Competitiveness p y$ 

The Role of Standards, Assessment and Credent ialing 



o a4 

ERIC 



Exhibit 8, page 2 of 2 (Great Britain - GNVQ) 

UNIT* 2M1 PRODUCE AND APPLY STANDARD COMMUNICATION 
METHODS 



Element U: Design a product from a given design brief 
Performance criteria: 

1 key design features are identified from a given design brief and accurately summarised 

2 production process constraints are identified from a given brief and accurately summarised 

3 relevant design and production constraints are identified and a specification is produced 

4 potential proposals are generated to meet the specification 

5 given evaluation criteria are used to select the final design proposal 

6 sufficient technical information is obtained to allow the key stages of production to be 
identified 

7 order and key stages of production are outlined 
Range: 

Key design features: aesthetic; contextual; functional; performance; production parameters (eg. cost; 
quantity; time) 

Production constraints: labour (eg skill, personnel); available materials; available technologies; quality; 
health and safety (ie limiting/enabling features of production processes that will have a major impact on 
the specification) 

Evaluation criteria relating to: frw design features (eg. aesthetic; contextual; functional; performance; 
production parameters); production constraints (eg> labour; available materials; available technologies; 
quality; health and safety) 

Technical information relating to available: materials/components; equipment/machinery; cost; 
quantity; time 

Key stages of production: pre-production; production; post production 



Evidence Indicator: 

Product design proposals for the manufacture of two products using contrasting materials and scales of 
production. Evidence of participation in constructive interactions which clarify and interpret the brief, 
establish a specification, generate alternative design proposals, and leading to a final evaluated proposal, 
detailing order and key stages of production. In addition the candidate should demonstrate an 
understanding of the implications of the range dimensions and key principles in respect of them. The 
unit test will confirm the candidates coverage of range. 



March 1993 
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Exhibit 9 (Scotland) 



1.1 MODULE DESCRIPTORS 

Modules are the basic units on which the National Certificate is operated. Each 
descriptor provides all the necessary information for teachers and lecturers to 
devise a programme of learning which will meet students' vocational, 
educational and personal needs and lead to the accreditation of their individual 
achievement. 

Table 1.1 Components of a Module Descriptor 



REFERENCE NUMBER and 
DATE 


This ensures that the correct module descriptor 
is being used. 


TITLE 


Thisgivesa clear idea of whatthe module is 
about. It is the module title which appears on 
the student's certificate. 


PURPOSE 


Thir gives a clear guide to the general changes 
in the learner which are to be brought about. 
An explanation is given of the uses for which 
the module was designed and the ways in 
which it can best be used in an educational or 
raining programme. The target group for the 
module is also specified here. 


PREFERRED ENTRY LEVEL 


This shows the level of previous achievement 
or experience without which it is likely that a 
student will have difficulty in successfully 
completing the module. 


LEARNING OUTCOMES AND 
PERFORMANCE CRITERIA 


These specify clearly the key competences 
which are to be accredited and to descrioe 
how satisfactory performance can be 
recognised. 


CONTENT/CONTEXT 


These give an indication to tutors of the range 
of contexts within which a module could be 
offered and the subject matter which would 
assist in the achievement of the Learning 
Outcomes. 


LEARNING AND TEACHING 
APPROACHES 


These suggest learning strategies which enable 
the Learning Outcomes to be achieved in as 
student-centred, participative and practical a 
way as possible. 


ASSESSMENT PROCEDURES 


These indicate the most appropriate way in 
which the Learning Outcomes can be assessed 
and give a description of the level of 
performance which must be reached if the 
Performance Criteria are to be met. 
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Exhibit 10, page 1 of 3 



(Great Britain) 



2. The National Standards for Assessment and Verification 



C 



t 



♦3 

; c 

i V> 

\ § 



a 



n. 



h 



aii Ay tt and tcxim wppew H* wt cwwww 1 tr— wit) »"* t»w » i« * > w' < » 



1 Ln 



{ AO Aytt p^pfitwi to' «*yf»oo*>g tf>» naming aoo otvttopfiwni tv«a«n 



1 »n# f Kit* 



A2l W*nWy im Curtfni CO<np«l«nc* o> <nO'v<Julll 



A22 Aflttt mot*<uari and group i pnem«t mm^g 



1 KI^ftviJIT^ vo 
rw*t*oj*i trw pant 



• ' Dn^n o<04n>ui«nai Vfvno 



| B> 1 Dt*t>op irairvng and o#v«com#Ai urai*o«i 



| 12 D»uo/i pri i»9^i to j 



■ j j\2 k»Wi<y ttiO'xe«i r»g-j<ftO mmp*»mini i mn»gic pun J 

■ i BO Sp tCl^*noo•f^lcf^lp^nl^m<^mttl^of9^n■M^■Onlll»ow^l^*^^y^g j 
- j B'4 Nfr$OHllt Iftfl IQrtf im lUOClOOnpT fttOufCai I 

J , B2^ totntrty ino agr— 'fi.fwftg and otvtlpQfntni ttrtitgm Wi; mmKlouft} rw.Oi j 

~ { S22 Ptl<gn waning prog'ammti wfticn matt Itinvog o»»pi | 

~j B23 Sotiry int ttwureti f>»wa lo^ltvtr pfo^timrtwi | 

"" } B2< Ttil. loitw 1^ ayt wa rrtng programma Oolqn* | 

( B2S AQf«« HimK^ pun! to p«hvtf tofrv^uiit iftfl pfQuDI" Q»»CW<» | 



to o*w»« »a-**o, »«a I— | 



I Ci' Ootam ratoutcti to o»ivt f ifufwg and otvtWpnwrrt piim 

• CO A«ocit* (fiourcti io tn* ttq«<'tm«nti ot tra»n<ng and o*wtoprn«'v. pUni 



t C2i P'tW« and p»«itni otmomirii*oni ano information and pro«tf« ae\«;t 10 
IWPOI warr.mg 



C22 Pftpift me prove* opponuttwi to* rnoiv*:vaii and Q'ovOt to Warn oy 
^ cowaooration 



j C23 Prtpsrf ir^ pf«vigt opponun^i«i Iot mdivcuail ang group* ID nntnagi tnt*> own 
teaming 



>C2 PfOvO* KJTvftg 03OO^V>«( J 

- a'* iuDPon to ***** <^»v4w- • 
| art i*o 0'Ovtt I 



i C24 Pttpa't ane pfoviot Itcnnpipgroapcnofnt warning rtioureti to tuopon 
i *naiw«uai and group W*w*ne 



C25 CoofQiftait tt>« ptcMfanonanflotwffy o* warning opponunnw* 



— I C26 A»iul and mpDOrt in* application ol Warning 



i— j C27 Onomaw warning lupppr. mat*r>an 



ri 



D ' IViUitrim.i'nrcl'- 



On Ewn 



and ift up irittmi tpr avamaung tnt trmning and oantepflwai lunciion 



— 1~- ( DO tv>k>an tnt tramtng and o«v*Ppm«nt tunwipn 



Di3 Mo<Wir iy»tam» andpraci<ti to tmp/ova tratrung *nc 04v»iopnw« 



0: Evwwiw ^ov«v* ino f'M 
0»KH» 



j 02i Pun and itt up lyittmi to tvatuaw tnt «cnwv# marf. ot oowoum 



— — j 023 EvalutiaPw acnwvwwm o< outoomti »oawm wamtng oO|<orm 



D» ModHy and ad*pt waning plant 



D3l' D^*i<)n■^gl!tfI* , . lot thi' coHi'dKin crfi-virttftto- 



'01 Av^i-.>JC«o»n*rtt6«put>K" 




036 idt*n]ity prt^viO^Uy .iCQutfixlcottipftoricp 



I Swocwn va<fwg »ii~l 
otytioenw* «d< j- 



tt Comooui* 10 KN»rc*i n 
in«w^ ana •»v*wa"wnt 



U PrpMM w luopon 

n<w^g and o*«*«09m«4 



Kg 



E 1 1 Cpntnoot* to tra m»ng a nd Otvtlopmtnt pracic* 



2 Evituiit ino o>»ittTwvitf atfvaneti m tr»m»ng ana d#v#lppm*t 



E21 Adm«ntH»iifa>n»ngtacAtwi 



£22 E»ta©*i»h and maintain mtovmanon lyittmi 



£23 tatawwh ane ma>m>tn hnanoii aytitmt 



Center for Learning and Competitiveness 

The Role of Standards, Assessment and Credentialing 



Page 78 



ERLC 



97 

BEST COPY AVAILABLE 



Exhibit 10, page 2 of 3 (Great Britain) 
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Exhibit 10, page 3 of 3 (Great Britain) 
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Exhibit 11 (Great Britain) 



• '•V"": " : / The Targets \ • 



Foundation Learning 

1. By 1997, 80% of young people 10 reach NVQ2 (or 
equivalent) . 

2. Training and education to NVQ3 (or equivalent) to be 
available to all young people who can benefit. 

3. By 2000, 50% of young people to reach NVQ3 (or 
equivalent). 

4. Education and training provision to develop self- 
reliance, flexibility and breadth. 

Lifetime Learning 

1. By 1996, all employees should take part in training or 
development activities. 

2. By 1996, 50% of the work-force to be aiming for 
NVQs or units towards them. 

3. By 2000, 50% of the work-force to be qualified to at 
least NVQ3 (or equivalent). 

4. By 1996, 50% of medium to larger organisations (200 
or more employees) to be "Investors in People". 
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Exhibit 12, page 2 of 3 (Denmark) 



VOCATIONAL EDUCATION AND TRAINING SYSTEMS IN DENMARK 



Combined education and training with two routes of 
entry 

Apprenticeships and efg have been replaced by a combined type of 
education and training with two entry routes. Trainees can begin em- 
ployment in an enterprise, or start a 20-week introductory period (first 
term) at a commercial or technical school. This is followed by a 20-week 
period (second term), and teaching alternates subsequently between 
practical work and school instruction. The two routes are of equal value, 
and the education and training provided is identical after the first 20 
weeks. 



ENTRY ROUTES 



Year 1 



Year 2 



Year 3 



Year 4 



Start 

employment 
in an 

enterprise -* 



Start in 
school 



Schooling 



Common 
.-^ school # 
period 



_L 



+ 20 weeks + 20 weeks + 
□ School □ Job training ■ Contract with employer 

An entire type of education and training can last up to five years, 
alternating between practical work and schooling, and with a maximum 
of 80 weeks of schooling. Most of the types, however, have a duration of 
3 or 4 years. The roughly 300 types of training which the former system 
covered have now been combined to achieve a smaller number - p.t. 86 
- with broad access, and late specialisation. There is free access and free 
choice of schools. 
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Exhibit 12, page 3 of 3 (Denmark) 
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Exhibit 13 (Denmark) 



Skills Needed by the Metalworker of the I990's 

* COMPREHENSION OF THE ELEMENTS, TERMINOLOGIES AND 
CONCEPTS OF PRODUCTION 

* CONTROL OF OWN WORK, INCLUDING PLANNING DISCIPLINES 

* COMPREHENSION OF PROPERTIES AND APPLICATION OF PRODUCTS 

* NEW TECHNOLOGY, NEW MATERIALS AND THE APPLICATION AND 
IMPLICATIONS OF NEW METHODS 

* INCREASED CO-OPERATION 

* PARTICIPATION IN DEVELOPMENT AND READJUSTMENT PROJECTS 

* KNOWLEDGE REQUIRED OF ADMINISTRATIVE SYSTEMS 

* LINGUISTIC REQUIREMENTS 

* EDB SKILLS AND COMPREHENSION REQUIRED 

* DRAWING COMPREHENSION 

* MEASURING TECHNIQUES 

* INFORMATION HANDLING REQUIREMENTS 

* COMPREHENSION OF OVERALL CONNECTIONS BETWEEN 
PRODUCTION AND COMPANY 

Source: Danish Metalworkers' Union 
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Exhibit 14 (Great Britain - GNVQ) 



STRUCTURE OF GNVQ3 



Mandatory 
Units 



Optional 
Units 



Art 8c Design 
Business 

Health 8c Social Care 
Leisure 8c Tourism 
M anuf acturing 



Core Skill Units, Level 3 



Communication 



Application of number 



Information Technology 
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Exhibit 15, page 1 of 3 (Great Britain - GNVQ) 



DEVELOPING AND PILOTING THE NCVQ CORE SKILL UNITS 
Element list for Communication 
Communication level 1 

1.1 Take part in discussions with kn^w individuals on routine matters 

1.2 Prepare written materials in pre-set formats 

1.3 Use images to illustrate points made in writing and in discussions with known 
individuals on routine matters 

1.4 Read and respond to written material and images in pre-set formats 

Communication level 2 

2.1 Take part in discussions with a range of people on routine matters 

2.2 Prepare written material on routine matters 

2.3 Use images to illustrate points made in writing and in discussions with a range of 
people on routine matters 

2.4 Read and respond to written material and images on routine matters 

Communication level 3 

3.1 Take part in discussions with a range of people on a range of matters 

3.2 Prepare written material on a range of matters 

3.3 Use images to illustrate points made in writing and in discussions with a range of 
people on a range of matters 

3.4 Read and respond to written material and images on a range of matters 

Communication level 4 

4.1 Take part in, and evaluate the effectiveness of, discussions with a range of people on a 
range of matters 

4.2 Prepare, and evaluate the effectiveness of, own written material on a range of matters 

4.3 Use and evaluate the effectiveness of own use of images to illustrate points made in 
writing and in discussions with a range of people on a range of matters 

4.4 Read and respond to written material and images and recognising the factors which 
influence own interpretation 

Communication level 5 

5.1 Lead, and evaluate the effectiveness of, discussions with a range of people on a range 
of masters 

5.2 Prepare, and evaluate the effectiveness of, own and others' written material on a range 
of matters 

5.3 Use and evaluate the effectiveness of own and others' use of images to illustrate points 
made in writing and in discussions with a range of people on a range of matters 

5.4 Read, and respond to written material and images recognising the factors which 
influence own and others' interpretations 
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Exhibit 15, page 2 of 3 (Great Britain - GNVQ) 



REPORT NO 16 



Element list for Application of number 
Application of number level 1 

1.1 Gather and process data using group 1 mathematical techniques 

1.2 Represent and tackle problems using group 1 mathematical techniques 

1 .3 Interpret and present mathematical data using group 1 mathematical techniques 

Application of number level 2 

2.1 Gather and process data using group 1 & 2 mathematical techniques 

2.2 Represent and tackle problems using group 1 & 2 mathematical techniques 

23 Interpret and present mathematical data using group 1 & 2 mathematical techniques 

Application of number level 3 

3.1 Gather and process data using group 1, 2 & 3 mathematical techniques 

3.2 Represent and tackle problems using group 1, 2 & 3 mathematical techniques 

3.3 Interpret and present mathematical data using group 1, 2 & 3 mathematical techniques 

Application of number level 4 

4.1 Gather and process data using group 1, 2, 3 & 4 mathematical techniques 

4.2 Represent and tackle problems using group 1, 2, 3 & 4 mathematical techniques 

4.3 Interpret and present mathematical data using group 1, 2, 3 & 4 mathematical 
techniques 

Application of number level 5 

5.1 Gather and process data using group 1, 2, 3, 4 & 5 mathematical techniques 

5.2 Represent and tackle problems using group 1, 2, 3, 4 & 5 mathematical techniques 

5.3 Interpret and present mathematical data using group 1, 2, 3, 4 & 5 mathematical 
techniques 
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Exhibit 15, page 3 of 3 (Great Britain - GNVQ) 



DEVELOPING AND PILOTING THE NCVQ CORE SKILL UNIT 



Element list for Information technology 

Information technology level 1 

1.1 Input data into specified locations 

1.2 Edit and organise information within individual applications 

1.3 Present information in pre-set formats 

1.4 Use operating routines which maximise efficiency 

Information technology level 2 

2.1 Set-up, use, and input data into storage systems 

2.2 Edit, organise and integrate information from different sources 

2.3 Select and use formats for presenting information 

2.4 Identify and use operating routines which maximise efficiency 

Information technology level 3 

3.1 Set system options and set up, use, and input data into storage systems 

3.2 Edit, organise and integrate complex information from different sources 

3.3 Select and use formats for presenting complex information from different sources 

3.4 Select and use applications when they are an effective way of working with 
information 

Information technology level 4 

4.1 Set system options, set up and use storage systems, and prepare and input data 

4.2 Set up and use automated routines to edit, organise and integrate complex information 
from different sources 

4.3 Set up and use automated routines to format and present complex information from 
different sources 

4.4 Evaluate and select applications for use by self 

Information technology level 5 

5.1 Set system options, set up and use storage systems, and validate, prepare and input 
data 

5.2 Investigate and resolve problems in editing, organising and in: grating complex 
information from different sources 

5.3 Investigate and resolve problems in formatting and presenting complex information 
from different sources 

5.4 Evaluate and select applications for use by self and others 
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Exhibit 16, page 1 of 3 (Denmark) 



Igennem irtusinder har bygmestre og h&ndvaerkere fiet og vidcrcgivct 
erfaringer om bestemte konstruktioners stabilitet. 

Hvis man byggcr en konstruktion som vist pi tegningerne, cr den ustabil: 




Det svarer helt til den viden, vi har: en firkant, hvis sidelaengder er kon- 
stante, kan have forskellige former. 



Bygger man derimod felgende konstruktion, er den stabil: 



Fra geometrien ved vi, at trekanter med samme sidelssngde har samme form. 
Dei er altsi ikke helt tilfaeldigt, n£r de optrasder i mange konstruktioner. 



Ls"'*2*- rs&rjt-i so-n efen uoYmrt* efr r r 5t* +3i>*ner/i es Projckt *f Firms mm* 
Mc*n^+r? op Tharjen, Grun o$ £*'ffift$*r. op Sites, s+rnf Hittrjps /*yt 




220f». rtrf t***+r 3**mm JLJm. IWi^t p**m 3tr »f rp * 't*mr* 

fX+OS cm. JO f* . 

i'**-* ow'm'r *» Srer.2M**€S*x>s. #/r cm. St* Area* /fSmntOSn^: mp *t cm* tfSm 
>-rc" r&"o* . 



Illustrationcrnc nedcrst side 208 cr 
gengivet cftcr »Hverdagskunst - vcr- 
dcnskun«tu af Broby Johanscn. For- 
lagct Fremad 1942 og cftcr »Dan$k 
siilhistorieu af H. H. Engqvist. 
Thanning og Appcl 1962. 
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Exhibit 16, page 2 of 3 (Denmark) 



Viden om geometri bruges ogsa i det mere fornejeiige, f.eks. bruger man 
viden om vinkler, nar man spiller billard. Spillet billard (eller: kegle-billard) 
er en sportsgren - Dansk Billard Union er optaget under Dansk Idrasts For- 
bund, og der afholdes bade danske og europaeiskc mesterskaber. 




cvT-i.-.i. ■•'\ttr 




Kort fortalt gar spillet ud pa, at man med en kfl stoder til en rod bal (kugle), 
som sa skal ramme en hvid bal - som sa igen skal ramme keglerne, den 
anden hvide bal eller ga i hul. 

Det er altsi vigtigt, at man kan beregne bal'ernes baner pa forhand. 
Se pa tegningen nedenfor: 




• Rod : al 

O Hvid ba! 




Kcgicr 



• • • 



Den stiplede lime viser den rode bal's bine, hvis den bliver 
rarnt przecist pi midten. 

Hver af de to gange, den rede bal rammer baiiden (kanten omkring bordet) 
kan man se, at vinklerne er lige store. 

Ved at fa baFerne til at skrue, kan man acndre vinklerne - og sa kratves 
der en ret stor geometrisk og fysisk erfaring for at kunne beregne bal'ernes 
baner. 
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Exhibit 16, page 3 of 3 (Denmark) 



En vinkcl kan vxrc: 




spids 
0°<v°<90° 



ret 
v° = 90° 




stump 
90°<v°< 180° 



v° 



iigc 
v° = 180° 



men den kan ogs& vaere storre 
end 180°: 



Et punkt pi en vinkelhalve- 
rings! inie bar samme af stand 
til vinklens ben. 



Topvinkler er lige store. 



Ensliggende v inkier ved pa- 
rallelle linier er ligt store. 
\l a| = |z- b| nkv og kun nir 
/II m. 





|Z-X|4.|^y| = 180° 
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Exhibit 17 (Denmark) 



»:R M8 Table for the dept's evaluation of apprentice igg^m n 

with a view to an eventual extra effort on weak l ^' JU ^ 
A245 3. sai Points 



Participants : 



Distribution: Apprentice/Dept/xxx/xxx 

Comnents in general concerning social behaviour, disciplin 



MT « Very good 



T = Good IT * Not good 

Trainee period Theoretical period 



Tasks || MT 


T IT 


MT 


T 


IT 






























i 




















































1 







































































































Points requiring follow-up 



Sign. Afd.L / VF 
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Exhibit 18 (Denmark) 



A202-245 



Evaluation after school - level 
period • 



•4« l 

KDT 



Distribution: Apprentice/xxx/xxx 



Name 



Date 



Set of marks from school period. See enclosure. 



2 



Did the school period meet with your expectations 

(tasks/quality) 



Are you confident that you achieved the optimum result of the 
teaching, if not at which stage do you think it slipped/you did not 
understand 



* Has the practical part of the teaching been as expected, if not, wher 
do you think it went wrong (did not work out very well) 



5 Sign: <L 



im.w.14 
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Exhibit 19 (Great Britain - NVQ) 



GUIDE TO NATIONAL VOCATIONAL QUALIFICATIONS 

NVQ Assessment model 

Elements of competence 
with 

Performance criteria 

determine form and amount of evidence to be collected 
through a combination of the following methods 




Performance evidence 

from 

♦ natural observation in 
the workplace 

♦ extracted examples within 
the workplace 

♦ simulations (competency tests, 
skills tests, proficiency tests, 
projects/assignments etc) 



Supplementary evidence 

from 

♦ oral questioning 

♦ open written answers 
(short, long, essays etc) 

♦ multiple choice tests 



Evidence from prior achievements 

reports, designs, computer 
programs, certificates from 
other sources etc. 
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Exhibit 20 (Scotland) 



LIST OF STANDARD INSTRUMENTS OF ASSESSMENT 



THE ASSESSMENT OF PRACTICAL COMPETENCE 

Practical Exercise 

Simulation 

Role Play 

Aurai/Oral 

Project 

Assignment 

Case Study 

Log Book 
Personal Interview 
Questionnaire 

THE ASSESSMENT OF COGNITIVE COMPETENCE 



SELF REPORT 
TECHNIQUES 



Project 
Assignment 
Case Study 
Written Test 



a) CONSTRUCTED RESPONSE 
Restricted Response 
Extended Response 
Structured Question 



FREE RESPONSE 
TESTS 



Short Answer 
Completion 

b) SELECTED RESPONSE 
Multiple Choice 
Multiple Response 
Matching 
Grid Question 
Alternative Response 
Assertion/Reason 



OBJECTIVE 
TESTS 
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Exhibit 21, page 1 of 2 (Scotland) 




SCOTTISH VOCATIONAL EDUCATION COUNCIL Form evsb 

EXTERNAL VERIFIER REPORT 
HOLD ON CERTIFICATION 



DO NOT SEND COPY TO CENTRE 



1 VERIFIER NAME 



NUMBER 



2 COGNATE GROUP NAME 



3 CENTRE NAME 



NUMBER 



NUMBER 



4 DATE OF VISIT 



5 MODULE/UNIT HELD 



6 REASONS FOR HOLD (tick as appropriate) 



NUMBER 



REASONS 



OUTCOMES 



1 2 3 4 5 6 



Assessment Instruments Not Available 



Inappropriate Assessment Instrument 



Incorrect or Inappropriate Assessment Specification 



No/Insufficient Evidence of Candidate Performance 



Performance Criteria Not Met 



Inappropriate Judgement of Candidate Performance 



No Record of Candidate Achievement 



Internal Standardisation Arrangements Unsatisfactory 



7 COMMENTS 
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Exhibit 21, page 2 of 2 (Scotland) 



8 CAN THE CENTRE RECTIFY THE PROBLEMS IDENTIFIED? 
(tick V as appropriate) If NO please give your reasons. 



YES NO 



9 WHERE THE DIFFICULTIES CAN BE RESOLVED, WHAT ACTION HAVE YOU 
RECOMMENDED THATTHE CENTRETAKES? (please give precise details) 



CENTRE CONTACT 



10 DATE BY WHICH THE CENTRE HAS AGREED 
TO HAVE THE PROBLEMS RESOLVED? 



SIGNED DATE 
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